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1 . SUMMARY 


1 . 1 Objectives 

This  report  covers  the  first  portion  of  a planned  three-year 
program  of  research  and  development.  The  entire  program  is  directed  toward 
the  use  of  adaptive  supervisory  computer  programs  to  improve  the  acquisition 
and  utilization  of  information  by  command  personnel  in  large-scale  systems 
for  command,  control  and  communication  (C3).  The  work  presented  here  deals 
specifically  with  the  application  of  adaptive  inul ti-attribute  utility  models 
to  dynamically  select  information  for  a system  user,  on  the  basis  of 
situational  requirements  and  his  observed  information  preferences.  Specific 
objectives  of  the  7-month  program  included. 

(1)  Analyze  design  principles  and  approaches  for  the  dynamic 
control  of  information  flow  in  C3  systems. 

(2)  Develop  and  implement  a prototype  adaptive  (individualized) 
information  selection  model. 

(3)  Demonstrate  and  evaluate  automatic  information  selection 
capabilities  in  a simulated  C3-t.ype  task. 

(4)  Establish  guidelines  for  application  of  information  selection 
models  to  higher-level,  multi-man  C3  systems. 

These  objectives  were  met  by  integrating  the  new  system  concept  with 
adaptive  modeling  technology  established  by  Perceptronics  under  previous 
ARPA- sponsored  programs.  In  particular,  the  present  adaptive  multi-attribute 
utility  model  for  information  selection  has  at  its  core  a trainable  utility 
estimator  previously  developed  for  computer  aiding  of  dynamic  decision 
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processes.  Similarly,  the  selection  model  was  applied  to  and  evaluated 
on  a modified  version  of  a simulated  ASW  submarine  tracking  task  used  in 
earlier  experimental  studies  of  decision  aiding. 

This  approach  allowed  us  to  demonstrate  successfully  a prototype 
adaptive  system  for  automatic  information  selection  in  a relatively 
realistic  C3  situation.  It  is  planned  that  the  present  model  will  be 
incorporated  into  a larger  system  of  adaptive  supervisory  computer  programs. 
Together,  these  computer  programs  will  compose  an  integrated  complex  of 
man-computer  models,  procedures,  and  aids  for  real-time  management  of 
information  flow,  in  concordance  with  specific  information  processing 
and  decision  making  requirements.  Our  analysis  and  empirical  results 
indicate  that  this  approach  can  produce  sizeable  reductions  in  decision 
time,  as  well  as  improvements  in  the  quality  of  the  information-based 
decisions. 

1 . 2 Technical  Approach 

1.2.1  Rational e.  Technical  advances  have  led  to  increases  in  the  speed, 
mobility,  and  destructive  power  of  military  operations.  The  amount  and 
rate  of  information  acquisition  has  increased  accordingly.  Information 
must  be  processed  more  efficiently  and  more  effectively  for  commanders 
to  make  tactical  decisions  responsive  to  the  rapidly  changing  succession 
of  events.  To  meet  this  need,  new  computer-based  systems  for  conmand, 
control,  and  communications  (C3)  are  being  developed  and  implemented.  These 
systems  are  intended  primarily  to  aid  in  the  collection,  processing,  and 
utilization  of  different  types  and  amounts  of  military  data.  The  overall 
process  is  cyclic  --  as  information  is  being  used,  other  information  is 
being  processed,  and  new  information  is  being  sought  and  collected.  The 
dynamics  of  information  flow  are,  therefore,  of  critical  importance  and 
must  be  constantly  monitored  and  directed. 
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The  consensus  concerning  current  computer-based  military  systems 
for  C3  operations  is  that  they  have  increased  the  rate  and  density  of 
information  flow  to  such  an  extent  as  to  overwhelm  a commander  and  his 
staff.  New  C3  techniques  are  required  to  control  information  flow  so 
as  to  best  match  system  capability  with  human  characteristics  in  the 
man-computer  interaction.  Our  review  of  previous  research,  presented  in 
Chapter  2,  suggests  that  a significant  step  in  this  direction  would  be 
to  individualize  and  automate  information  selection.  This  would  allow 
each  system  user  continuously  to  obtain  information  that  is  both  relevant 
and  timely  with  regard  to  his  individual  processing  characteristics  and 
immediate  decision  making  needs.  Considering  the  large  number  of  interrelated 
users  in  a typical  C3  system,  the  effect  on  total  system  performance  would 
be  to  substantially  increase  throughput  while  also  improving  decision  making 
qual ity. 

1.2.2  System  Concept.  The  basic  concept  of  the  model-based  selection 
system  is  illustrated  in  Figure  1-1.  The  message  universe  includes  all 
information  potentially  available  to  the  recipient,  or  system  user.  In 
the  manual  mode,  the  recipient  continuously  selects  messages  in  accord 
with  a selection  strategy.  A strategy  represents  individual  preference 
for  information  in  response  to  situational  needs.  In  the  automatic  mode, 
an  adaptive  information  selection  mechanism  automatically  supplies  the 
user  with  information  on  the  basis  of  his  individual  selection  strategy. 

The  factors  which  characterize  an  individual's  strategy  are 
incorporated  in  an  adaptive  multi-attribute  utility  model.  In  this  model, 
incoming  information  is  decomposed  into  measurable  attributes.  Attribute 
levels  of  a message  are  determined  by  vectors  which  include  both  situational 
requirements  and  source  characteristics.  The  subjective  weight,  or  utility, 
that  the  user  places  on  each  attribute  is  estimated  on-line,  by  an  adaptive 
technique,  as  the  user  manually  selects  information.  The  utilities,  in 
combination  with  the  measured  attribute  levels,  permit  calculation  of  a 
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multi-attribute  utility  ( MAU ) value  for  each  incoming  message.  The  selector 
mechanism  then  passes  to  the  recipient  those  messages  with  high  MAU,  that 
is,  of  high  value  to  him.  The  selector  can  also  use  the  MAU  values  to 
improve  an  individual's  information  gathering  efficiency.  For  example, 
most  users  select  more  information  than  is  actually  necessary  to  reach  a 
decision.  The  model-based  selector  can  reduce  the  transmitted  message  set 
by  eliminating  those  messages  which  contribute  less  than  some  criterion 
value  of  utility.  In  the  present  study,  for  example,  we  examined  a pruning 
rule  which  ranked  messages  in  order  of  decreasing  MAU,  and  eliminated  those 
messages  for  which  the  MAU  was  less  than  15%  of  the  total  MAU  of  previously 
selected  messages. 

1.2.3  Demonstration  and  Evaluation.  The  methodology  used  to  implement 
this  system  concept  is  described  in  Chapter  3.  For  purposes  of  demonstration 
and  evaluation,  the  adaptive  information  selection  model  was  applied  to 
a simulated  ASW  tracking  task.  This  task  requires  the  operator  to  track 
continuously  the  movements  of  a submarine  and  a whale  over  a segmented 
expanse  of  ocean.  The  probable  locations  of  the  tracked  objects  are 
given  by  a computer-generated  intelligence  report.  The  operator  uses 
this  report  to  select  a set  of  sensors,  or  information  sources,  for 
distribution  over  the  ocean  locations  of  interest.  The  available  information 
sources  differ  in  cost,  reliability,  discriminabil ity,  etc.  On  the  basis 
of  the  information  gathered,  he  reports  the  present  location  of  the  objects. 
His  status  report  triggers  a new  intelligence  report  for  the  next  cycle, 
and  the  task  continues. 

In  the  automatic  mode,  the  process  of  source  selection  is  taken 
over  by  the  adaptive  model,  and  the  operator  works  entirely  with  this 
dynamically  selected  information.  The  task  provides  a realistic  information 
environment,  a large  number  of  selection  decisions,  and  objective  measures 
of  decision  performance  (tracking  accuracy  and  information  cost  expenditure). 
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The  model-based  selection  system,  the  simulated  task,  and  the  associated 
performance  measurement  programs  were  supported  on  an  Interdata  70 
minicomputer  with  48K  words  of  memory.  The  operator  interacted  with  the 
system  through  a graphics  terminal  and  keyboard. 

1 . 3 Findings 

System  evaluation  included  a structured  study  of  model  performance 
and  dynamic  behavior,  and  a systematic  empirical  examination  of  operator 
task  performance  with  varied  types  of  automatic  information  selection. 

The  results,  presented  in  Chapter  4,  are  summarized  below. 

1.3.1  Model  Performance.  In  brief,  the  use  of  an  adaptive  multi-attribute 
utility  model  for  information  selection  was  successfully  demonstrated. 
Specific  observations  of  model  performance  included: 

(1)  Information  Attributes.  Seven  attributes  were  sufficient 
to  characterize  the  ASW  information.  These  were  arrived  at 
through  a reiterative  process  of  analysis  and  empirical  test. 

(2)  Utility  Convergence  and  Adaptation.  Utilities  for  attributes 
converged  rapidly  to  reflect  consistent  individual  strategies 
of  information  selection.  Typically,  convergence  took  10-20 
adjustment  cycles,  spanning  a few  task  trials.  Changes  in 
strategy  produced  corresponding  changes  in  the  selection 
behavior  of  the  model. 

(3)  Individualized  Message  Sets.  Automatic  selection  of  information 
yielded  distinct  message  distributions  for  distinct  individual 
strategies. 


(4)  Information  Filtering.  Overall,  the  adaptive  system 

was  able  to  select  automatically  about  5 to  7 highly  preferred 
messages  from  a potentially  available  universe  typically 
in  the  order  of  78,000  messages,  a filtering  ratio  of  over 
15,000  to  1 . 

1.3.2  Operator  Performance.  Empirical  evaluations  showed  that  an 
operator  was  able  to  perform  the  tracking  task  successfully  with  automatic 
selection  of  information  sources  by  the  adaptive  model.  Comparison  of 
different  automatic  selection  modes  revealed: 

(1)  Preferential  Strategies.  ASW  tracking  performance  was 
better  when  information  selection  included  utility  criteria 
(individualized  attribute  weights)  than  when  utility  criteria 
were  eliminated  from  the  selection  procedure  (uniform  attribute 
weights) . 

(2)  Reduced  Information  Set.  ASW  tracking  was  further  improved 
when  relatively  low-utility  information  was  dynamically 
pruned  from  the  individualized  information  set.  The 
effectiveness  ratio  between  performance  with  the  reduced 
and  with  the  non-reduced  set  was  about  1.5  to  1.  The 
effectiveness  between  performance  with  the  best  pruned 
strategy  and  with  the  uniform-weight  strategy  was  1.8  to  1. 

(3)  Acquisition  Time.  Informal  observation  indicated  that 
automatic  information  selection  markedly  reduced  the  time 
required  for  manual  information  acquisition  in  the  simulated 
ASW  task.  Time  reduction  was  in  the  order  of  50  to  1. 
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1.4 


Appl ications 


1.4.1  Domain.  The  domain  of  application  for  the  adaptive  selection 
model  developed  here  is  a dynamic  local  environment,  where  new  information 
of  the  same  general  type  must  be  processed  repeatedly.  Such  environments 
are  ubiquitous  in  modern  computer-based  command  and  control  operations.  If 
the  man  and  the  computer  can  be  considered  as  representing  a single  system, 
then  the  goal  of  the  technique  is  to  provide  the  man  with  information 
which  will  improve  the  overall  decision  output  of  the  system. 

Within  its  domain  of  application,  the  mul ti-attribute  utility 
approach  is  highly  general izabl e.  The  present  demonstration  is  indicative, 
since  the  ASW  test  bed  was  not  specially  tailored  for  the  model,  as  is  often 
the  case.  In  addition  to  generality,  other  advantages  of  the  multi- 
attribute utility  formulation,  discussed  in  Chapter  5,  include  parsimony, 
robustness,  speed  of  adaptation,  flexibility  and  versatility. 

1.4.2  Supervision  of  C3  Information  Flow.  The  present  demonstration 
dealt  with  a single  information  user.  However,  we  can  consider  the  typical 
C3  system  as  a hierarchical,  multi-level  arrangement  of  users.  People  at 
one  level  process  information  for  people  at  the  next  level,  collecting 

and  integrating  data  until  a decision  commensurate  with  their  level  can 
be  made.  Thus  each  person  in  the  structure  is  at  times  a user  of  information, 
at  times  a source  of  unprocessed  or  processed  information,  and  at  times  a 
source  of  decisions  passed  to  higher  levels  of  the  hierarchy.  A matrix 
analysis,  presented  in  Chapter  5,  suggests  that  optimum  information  flow 
in  such  a structure  could  be  controlled  by  a supervisory  program  incorporating 
both  heuristic  control  algorithms  which  are  situation-dependent,  and  a set 
of  behavioral  models,  which  depend  on  psychological  constructs  and  on 
individual  user  characteristics.  Among  the  most  significant  models  will 
be  those  which  define: 
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(1)  Multi-Attribute  Utility  for  Information 

(2)  Information  Routing 

(3)  Information  Pacing  and  Load 

Feasibility  of  the  first  model  has  been  demonstrated  by  the  present  study. 
The  preliminary  analysis  of  Chapter  5 indicates  that  the  multi-attribute 
technique  lends  itself  to  important  aspects  of  the  other  two  models  as 
well.  It  is  planned  to  explore  this  approach  in  our  extension  of  the 
current  work  to  a supervisory  system  of  information  control,  which  will 
include  the  pacing  and  routing  functions. 
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2.  SELECTIVE  LITERATURE  REVIEW 
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2.1  Purpose 

The  purpose  of  this  survey  and  analysis  of  relevant  literature  is 
to  identify  design  principles  for  models  of  information  flow  in  C3  systems. 
Many  experiments  have  investigated  information  seeking  and  decision  making 
in  relatively  wel 1 -structured  situations.  The  results  of  these  experiments 
provide  much  useful  data  on  how  people  in  C3  systems  seek  and  use 
information.  The  review  below  attempts  to  demonstrate  that  these  data 
support  the  development  of  on-line  computer  models,  particularly  adaptive 
ones,  designed  to  help  command  personnel  in  the  acquisition  of  appropriate, 
timely,  and  individually-suited  information. 

2.2  Organization  of  Literature 

A crude  but  generally  applicable  schematic  for  information  flow 
within  a C3  system  is  provided  below  (adapted  from  Lin  and  Garvey,  1972). 


The  diagram  identifies  the  major  phases  of  information  communication  and 
indicates  an  executive  structure  to  govern  the  flow  of  information.  The 
discussion  of  topics  to  follow  focuses  on  human  performance  in  specifying 
information  needs,  and  in  acquiring  and  utilizing  information  for  decision 
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making.  Throughout  the  review,  emphasis  is  placed  on  the  rationale  for 
developing  computer-administered  procedures  to  adaptively  present  automatically 
selected  information  todecision  makers. 
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2.3  Information  Requirements 

I 

2.3.1  Specification  of  Information  Requirements.  In  the  context  of  management 
information  systems,  Ackoff  (1967)  has  implied  that  the  amount  and  type  of 
information  that  a manager  thinks  he  needs  is  often  not  in  line  with  what  he 
actually  does  need  for  effective  decision  making.  In  general,  the  more 
deficient  the  manager's  " mental  model"  of  the  decision  situation,  the  more 
information  he  will  want.  The  result  may  be  that  the  manager  becomes 
overwhelmed  from  an  overabundance  of  irrelevant  information.  Ackoff  further 
states  that  "one  cannot  specify  what  information  is  required  for  decision 
making  until  an  explanatory  model  of  the  decision  process  and  the  system 
involved  has  been  constructed  and  tested". 


A similar  point  of  view,  concerning  military  intelligence,  has  been 
experessed  by  Williams  (1972): 


"Information  collectors  must  know  what  information  the  commander 
needs.  Too  often  he  does  not  tell  them.  Too  often  he  does  not 
know  himself  and  too  often  the  intelligence  people  are  not 
qualified  to  anticipate  for  him.  The  commander's  guidance  is  the 
vital  pulse  that  should  trigger  a meaningful  collection  effort. 
Unfortunately , ...  experience  indicates  that  many  commanders  leave 
this  responsibility  almost  entirely  to  their  G2s. " 


Two  important  implications  are  evident  from  these  remarks.  First,  a commander 
may  be  incapable  or  at  least  unwilling  to  accurately  specify  his  information 
needs  in  a particular  situations;  for  example,  it  would  not  be  entirely 
uncommon  for  a commander  to  take  the  easy  way  out  and  tell  his  collectors  to 
"get  all  the  information  you  can  get".  Second,  a commander' s reliance  on 
his  collectors  to  supply  the  appropriate  amount  of  relevant  information  may 
not  be  justifiable. 
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The  hypothesis  that  potential  users  of  information  systems  consider 
"everything"  critical  has  received  support  from  interesting  empirical  research 
conducted  at  ARI.  In  one  study  (McKendry,  Wilson,  Mace,  and  Baker,  1973), 
staff  officers  responded  to  survey  questionnaires  in  which  they  were  presented 
with  a tactical  mission  and  had  to  check-off  items  of  information  (combat 
events)  which  were  perceived  as  most  important  to  the  successful  completion 
of  the  mission.  An  important  finding  was  that  many  subjects  were  unable  to 
limit  themselves  to  30  crucial  events  (as  instructed)  from  among  a pool  of 
60  available  events.  In  a second  study,  experienced  field  officers  performed 
offensive  and  defensive  tactical  planning  within  a computer-controlled 
simulated  scenario  (Strub  and  McConnaughey , 1974).  The  officers  were 
permitted  to  request  information  at  will  from  a hierarchically-structured 
data  base,  and  the  computer  kept  track  of  each  request  noting  the  category 
of  information  (e.g.,  G-2  Intelligence)  and  level  of  detail  (e.g.,  enemy 
situation)  requested.  In  comparing  results  from  the  two  research  methods, 

Strub  (1977)  found  that  less  information  (in  terms  of  amount  and  level  of 
detail)  was  actually  requested  and  used  in  laboratory  exercises  that  was 
specified  in  questionnaires  as  being  essential  and  of  "should  be  requested" 
merit. 

Because  of  the  importance  of  this  last  finding  by  Strub,  an  attempt 
should  be  made  to  explain  it.  One  possibility  is  that  the  information  user, 
or  supplier  for  that  matter,  does  not  want  tn  he  caught  short  and  therefore 
overstates  his  information  needs.  The  conservative  bias  is  similar,  in 
principle,  with  the  typical  observation  that  subjects,  when  performing  in 
diagnostic  tasks,  purchase  more  information  than  recommended  by  normative 
Bayesian  procedures  (e.g.,  Levine,  Samet,  and  Brahlek,  1975).  Another  possible 
explanation  is  suggested  by  the  previously  referenced  remarks  of  Ackoff  (1967). 
Since  the  user  does  not  generally  construct  a model  of  the  decision  situation 
until  he  is  actually  engaged  in  it,  he  will  be  inclined  to  overstate  his 
needs  when  asked  to  anticipate  them  prior  to  confronting  the  problem  situation. 
Indeed,  in  the  ARI  studies,  survey-questionnaire  subjects  (i.e.,  those  who 
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expressed  a need  for  relatively  more  information)  projected  needs  across  a 
problem  context  described  in  terms  of  a hypothetical  tactical  situation; 
whereas  laboratory-experiment  subjects  (i.e.,  those  who  requested  relatively 
less  information)  experienced  actual  needs  while  performing  a realistic, 
although  simulated,  tactical  exercise. 

Additional  support  for  the  hypothesis  that  decision  makers  tend  to 
overstate  their  information  needs  is  available  from  Schroeder,  Driver  and 
Streufert  (1967).  They  included  a measure  of  information  satisfaction  within 
their  studies  of  information  processing  in  the  Tactical  and  Negotiations 
Game  (TNG).  The  TNG  is  a game  simulation  in  which  deci sion-making  teams 
are  given  the  task  of  directing  the  military,  economic,  intelligence,  and 
negotiation  activities  of  a small  underdeveloped  nation  plagued  by  an  internal 
revolution.  In  one  experiment,  the  effects  of  information  load  (varied  in 
terms  of  number  of  dimensions  of  information  presented  in  a given  time  span, 
diversity  of  the  information,  and  number  of  alternatives  that  each  unit  of 
information  added)  on  the  level  of  information  processing  and  decision 
performance  were  assessed.  At  the  conclusion  of  each  game  period,  subjects 
were  required  to  express  their  preferences  for  receiving  a different  amount 
of  information  relative  to  what  was  actually  received  in  the  previous  session. 
All  subjects  showed  a consistent  but  unjustifiable  bias  for  having 
considerably  more  information.  Subjects  even  ask  for,  or  say  they  would 
prefer,  more  information  following  periods  when  their  information  processing 
level  is  already  depressed  by  superoptimal  information  load.  Apparently, 
people  are  not  sufficiently  sensitive  to  reverses  in  load  which  are 
detrimental  to  information  processing. 


2.3.2  Balancing  the  Information  Supply.  As  a mechanism  for  summarizing  the 
problem  of  information  exchange  in  a field  Army  C3  system.  Baker  (1973)  has 
employed  the  "economic  man"  concept: 


I 
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"Observation  of  fie  id  exercises  hive  led  to  the  hype  thesis  that  the 
G2  produces  and  stox'es  more  data  than  the  G3  consumes  in  the  normal 
course  of  operations.  Likewise,  G3's  appear  to  ask  for  data  that 
G2's  have  not  yet  produced.  If  one  considers  this  situation  from  the 
standpoint  of  an  economic  analogy,  it  is  not  good  business  to  use 
resources  to  produce  items  for  which  there  is  no  buyer.  Contrariwise, 
if  a buyer  desires  something,  it  is  good  business  to  have  it  available 
to  sell  him.  In  the  dynamics  of  the  GI/G3  operations  the  G2  can  be 
likened  to  a producer  (of  information)  and  the  G3  as  a consumer 
( informa tion  user ) . " 

The  trick,  of  course,  is  to  maintain  an  appropriate,  cost-effective  balance 
of  information  supply  and  demand.  Focusing  upon  the  information  consumer, 
it  is  clear  that  if  too  little  information  is  disseminated  to  him,  he  cannot 
get  the  job  done;  on  the  other  hand,  if  he  gets  too  much  information,  he 
becomes  overloaded  and  must  expend  valuable  time  screening  items  for  relevant 
information. 

The  commander  in  many  C3  systems  appears  to  suffer  more  often  from  an 
overabundance  of  irrelevant  information,  much  of  which  he  did  not  ask  for, 
and  therefore  two  critical  functions  of  the  system  become  the  filtration  (or 
evaluation)  and  condensation  of  information.  However,  as  Ackoff  (1967) 
points  out,  the  literature  on  information  systems  seldam  refers  to  those 
functions  let  alone  considers  how  to  carry  them  out.  If  techniques  can  be 
developed  to  select  information  for  individual  users  so  that  each  gets  the 
information  that  is  most  relevant  and  useful  for  his  needs,  then  the 
techniques  would  simultaneously  carry  the  potential  to  accomplish  information 
filtration  and  condensation.  In  other  words,  the  process  would  aid  in 
weeding-out  the  information  which  is  not  desired  by  the  user. 

2.3.3  I ndvvi dualized  Adaptive  Selection.  As  data  come  into  a C3  system,  it 
must  be  determined  which  of  the  separate  users  should  be  the  recipients  of 
messages  containing  specified  classes  of  information.  In  a manual  system, 
the  selection  procedures  arise  in  response  to  specifically  stated  user 
requests  and  by  the  initiative  of  support  personnel,  who  through  training 
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and  experience  become  aware  of  the  information  needs  of  various  system  users. 
In  order  to  select  information  within  an  automated  system,  a set  of 
programmable  rules  is  necessary.  However,  as  inferred  from  the  research 
reviewed  above,  it  is  not  sufficient  --  because  of  inherent  human  biases  -- 
to  rely  solely  on  direct  user  statements  or  on  supplier  judgments  to  determine 
selection  specifications. 

Fortunately,  an  automated  C3  system  lends  itself  to  a more  promising 
approach  for  determining  routing  procedures.  Namely,  an  on-line  computer 
model  could  be  used  to  observe  and  track  each  individual's  information 
processing  behavior  and  thereby  dynamically  learn  and  assess  his  personal 
utilities  for  specific  types  of  information.  The  overriding  rationale  here 
is  that  since  the  decision  maker  can  attend  to  only  a few  information 
dimensions  among  many  (Hayes,  1964),  especially  when  under  pressure  (Wright, 
1974),  adaptive  modeling  can  selectively  choose  and  present  those  few 
dimensions  that  are  most  useful  to  him. 

The  notion  of  an  adaptive  system  for  providing  information  in  a C3 
system  is  further  supported  by  writings  of  Thompson  (1964,  1967).  He  dwells 
at  length  on  the  important  role  of  situational  context  and  command  context 
in  determining  information  relevance,  as  well  as  on  their  interaction  and 
impact  on  command  decision  making.  According  to  his  conceptualization, 

"...  th<-  data  ba  s,  ia  dyruunio  and  r».  s '•*.  rather  that  iv. 

It  is  bhi  < mb  diment  f t hi  current  concerns  of  particular 

headqx  ;r! , >rs.  Thus,  rather  t . • striving  for  bji  'tint  miner  set  ity , 

it  strives  for  relevance.  his  stetns  fron  thi  naturi  of 

the  data  f-tiu  'f  r ! /•••  • t <nta>  ■,  f •hang, 

Thompson's  theoretical  framework  has  received  empirical  support  from  recent 
studies  of  information  needs  and  priorities,  as  affected  by  such  independent 


variables  as  intensity  of  war  (Coates  and  McCourt,  1976)  and  the  user's 
staff  element  (e.g.,  G2,  G3)  and  echelon  of  command  (e.g.,  Army,  Corps, 
Division)  (McKendry,  et  a!.,  1973).  It  can  be  expected,  therefore,  that 
certain  characteristics  of  the  military  situation  and  aspects  of  user 
identification  could  serve  as  useful  input  to  any  information  selection 
model . 

2.4  Information  Acquisition 

2.4.1  Process  Description.  In  their  recent,  comprehensive  review  of  the 
information  processing  and  decision  making  literature,  Nickerson  and  Feehrer 
(1975)  identify  "information  gathering"  as  one  of  the  principal  tasks  to  be 
performed  in  a decision-oriented  system.  They  describe  the  process  as 
follows: 


"From  the  point  of  view  of  the  decision  maker , most  decision  situations 
are  characterized  by  some  degree  of  uncertainty . This  uncertainty  may 
involve  the  current  "state  of  the  world",  the  decision  alternatives 
that  are  available , the  possible  consequences  of  selecting  any  given 
one  of  them,  and  even  the  decision  maker's  preferences  with  respect 
to  the  possible  decision  outcomes.  One  of  the  major  problems  facing 
the  decision  maker , therefore , is  that  if  acquiring  information  in 
order  to  reduce  his  uncertainty  concerning  such  factors , thereby 
increasing  his  chances  of  making  a decision  that  will  have  a desirable 
outcome. 

"What  makes  the  problem  interesting , ih<J  nontrivial,  is  the  fact  that 
information  acquisition  can  be  costly , both  in  terms  of  time  and 
money.  Therefore,  the  decision  mak. •?*  ’ ist  determine  whether  the 
value  of  the  information  that  could  be  obtained  through  any  given 
data- collection  effort  is  likely  to  be  greater  than  the  cost  of 
obtaining  it.  And  therein  lies  a decision  problem  in  its  own  right.  " 


2.4.2  Experimental  Studies.  Much  research  has  been  done  on  information 
acquisition  in  decision  tasks,  but  most  studies  have  concentrated  on 
information  purchasing  behavior.  The  latter  have,  for  the  most  part,  failed 
to  capture  the  complexify  of  the  problem  that  often  faces  the  information 
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seeker  in  the  real  world.  In  the  typical  information  purchasing  experiment, 
information  from  a single  source  is  presented  to  the  subject,  with  his  task 
essentially  to  decide  whether  it's  worth  what  it  will  cost  to  acquire  it. 
However,  in  many  practical  situations,  the  decision  maker  or  commander  must 
seek  and  locate  the  information  he  needs  or  wants,  and  he  must  often  select 
from  among  alternatively  available  information  sources  of  differential 
quality  in  j^gis  of  information  diagnosticity  and  cost.  A few  relevant 
studies  ^^irnormation  selection  behavior  have  been  conducted  and  their 
findings  are  reviewed  below. 

^ Kanarick,  Huntington  and  Peterson  (1969)  studied  performance  in  a 
simulated  scenario  in  which  the  subject  had  to  reach  a binary  decision  about 
whether  an  enemy  submarine  was  either  present  or  absent  in  a given  vicinity. 

On  each  trial  the  subject  could  purchase  data,  from  one  of  three  different 
information  sources.  The  sources  varied  in  both  reliability  and  cost:  the 

higher  the  reliability  (diagnosticity)  of  the  source,  the  greater  the  cost 
for  consulting  it.  The  penalties  for  incorrect  decisions  were  also  manipulated 
experimentally.  Subjects'  behavior  was  sensitive  to  the  variations  of  the 
independent  variables;  however,  performance  was  deficient  when  compared  with 
an  optimal  Bayesian  model.  For  example,  they  consulted  the  most  reliable 
(and  most  costly)  sources  less  frequently  and  the  less  reliable  (less  costly) 
sources  more  frequently  than  they  should  have.  Also  they  generally  purchased 
less  information  than  required  by  the  optimal  model.  This  last  result  might 
be  accounted  for  by  the  common  finding  of  recent  research  that  subjects 
tend  to  over-estimate  the  diagnostic  impact  of  less  than  perfectly  reliable 
data  (e.g.,  Johnson,  Cavanaugh,  Spooner,  and  Samet,  1973). 


Although  Kanarick,  et  al.,  allowed  subjects  to  choose  amonq  multiple 
information  sources,  the  sources  were  presented  in  parallel.  However,  in 
real  life  situations,  information  from  various  sources  is  frequently  sought, 
generated,  and  made  available  in  a sequential,  rather  than  in  a parallel, 
mode.  In  a dynamic  situation,  furthermore,  the  uncertainty  of  the  environment 


may  force  the  information  seeker  to  perform  under  fluctuations  and  restrictions 
in  the  amount  of  available  information  and/or  the  level  of  resources  needed 
to  acquire  the  information.  For  example,  the  military  commander  can  take 
advantage  of  all  the  patrol  units  that  he  can  spare  but  still  require  more 
information  about  the  enemy. 

To  study  information  seeking  behavior  under  these  kinds  of  conditions, 
an  experiment  was  conducted  by  Levine,  Samet,  and  Brahlek  (1975).  These 
investigators  required  subjects  to  determine  which  of  four  populations  was 
being  sampled  in  a multinomial  Bayesian  task,  tach  sequentially  drawn  datum 
was  described  on  one  of  three  dimensions  which  represented  different  levels 
of  information  source  diagnosticity  (high,  medium,  and  low).  On  each  trial, 
the  subject  purchased  knowledge  of  the  identity  of  the  information  source 
which  was  available  on  that  trial,  and  he  had  the  option  to  either  purchase 
the  associated  datum  at  a fixed  additional  cost  or  pass  it  up  at  no  additional 
cost.  Using  this  paradigm,  the  amount  of  information  potentially  available 
and  the  percentage  of  it  which  could  be  purchased  by  the  resources  provided 
were  varied  factorial ly,  and  the  effects  on  information  selection  and 
purchasing  behavior  were  assessed.  The  principal  relevant  findings  were 
that:  (a)  relative  to  the  low  diagnostic  source,  subjects  purchased 

information  about  5 times  as  often  from  the  medium  and  high  diagnostic  sources; 
(b)  when  more  information  was  potentially  available,  subjects  were  more 
efficient  --  relative  to  an  optimal  Bayesian  model  --  in  selecting  from  among 
information  sources;  (c)  significantly  more  information  was  sought  as  both 
amount  of  available  information  and  purchasing  resources  increased;  and  (d) 
across  all  experimental  conditions,  subjects  generally  purchased  more 
information  than  was  recommended  by  the  normative  model.  With  regard  to  the 
last  finding,  the  subject  was  actually  paying  for  information  which  had  a 
negative  value,  i.e.,  information  whose  acquisition  led  to  a decrease  in 
expected  payoff. 


In  another  study  involving  the  selection  of  information,  Rapoport, 
Lissitz,  and  McAllister  (1972)  investigated  the  search  behavior  of  subjects 
required  to  find  a single  object  hidden  in  one  of  four  distinguishable 
locations.  For  each  location,  they  were  given:  (a)  the  t priori  probability 

that  the  object  would  be  detected  there;  (b)  the  probability  that  the  object 
would  not  be  found  by  a search  (i.e.,  a random  sampling);  arid  (c)  the  per- 
trial  cost  of  search.  In  agreement  with  the  previous  studies,  the  results 
indicated  that  subjects  do  not  consult  information  sources  in  an  optimal 
fashion.  Of  particular  interest  was  a finding  suggesting  individual 
differences  in  search  strategies;  that  is,  some  of  the  subjects  deviated 
from  the  optimal  policy  in  the  direction  of  maximizing  detection  probability, 
whereas  others  deviated  in  the  direction  of  minimizing  search  costs. 


The  results  of  experiments  on  information  purchasing  behavior  also 
relate  to  man's  capability  as  an  information  selector.  The  typical 
experimental  paradigm  allows  the  subject  on  each  trial  the  option  of  either 
purchasing  more  data  relevant  to  the  decision  that  he  is  required  to  make. 


or  to  stop  data  collection  and  make  a decision.  S topp i g data  collection 
also  a decision  and  defines  the  selection  of  a predec isional  information 
set.  The  various  studies  have  shown  that  subjects  are  highly  sensitive  to 
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informational  and  situational  parameters,  e.g.,  environmental  variance 
(Schroeder  and  Benbasat,  1975),  a priori,  probabilities  for  decision 
alternatives  (Green,  Halpert,  and  Minas,  1964),  data  di agnostici ty  (Snapper 
and  Peterson,  1971),  source  reliability  (Levine  and  Samet,  1973),  and  costs 
and  payoffs  (Pitz  and  Reinhold,  1968;  O'Connor,  Peterson,  and  Palmer,  1972), 
but  their  performance  departs  systematically  from  optimal  performance.  In 
general,  it  appears  that  too  little  information  is  purchased  when  much  is 
required  by  a Bayesian  Model  and  too  much  information  is  purchased  when  little 
is  required.  For  example,  subjects  have  been  found  to  require  from  two  to 
nine  data  observations  to  revise  their  opinions  as  much  as  Bayes'  theorem 
would  prescribe  for  one  observation  (Peterson,  Schneider,  and  Miller,  1965; 
Phillips  and  Edwards,  1966). 
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2.4.3  Rationale  for  Aiding  Information  Selection.  It  is  evident  that  when 
information  quantity  is  held  constant,  an  improvement  in  information  quality 
leads  to  an  improvement  in  decision  performance  (e.g.,  Levine  and  Samet, 

1973;  Snapper  and  Peterson,  1971).  With  regard  to  military  information 
processing  systems,  the  issue  has  been  stated  as  follows: 

"The  key  to  competent  decision  making  is  the  availability  of  current 
and  accurate  information.  It  is  not  the  quantity  of  information 
which  is  important.  Rather,  it  is  the  process  of  selecting  the 
pertinent  information,  ascertaining  its  significance , and  displaying 
it  in  a readily  understood  format  which  facilitates  the  decision- 
making process."  (Albright,  1975) 

Since  one  way  to  achieve  an  increase  in  information  quality  is  to  be  more 
selective  in  collecting  information,  we  can  ask  how  well  man  does  as  an 
"information  selector"  or  discriminator  among  alternative  information 
sources.  The  basic  conclusion  reached  by  each  of  the  experiments  reviewed 
above  is  that  although  subjects  are  sensitive  to  the  differences  in  information 
source  quality,  they  perform  poorly  in  selecting  among  information  sources. 

If  man  is  sensitive  to  key  informational  and  situational  parameters, 
why  does  he  consistently  show  systematic,  sterotypical  biases  when  choosing 
among  available  information  for  decision  making?  Apparently,  because  of  his 
limited  memory,  attention,  reasoning,  and  computational  capabilities,  he  is 
unable  to  integrate/aggregate/combine  various  dimensions  of  information  -- 
each  with  its  associated  graded  level  --  to  arrive  at  a compositive, 
subjective  value  for  the  information  which  is  consistent  and  valid.  Thus, 
for  example,  he  is  unable  to  appropriately  trade  off  the  reliability  of 
information  and  its  cost  (Kanarick,  et  al.,  1969).  Therefore,  it  appears 
that  a valuable  type  of  aid  for  a decision  maker  would  be  one  which  helps 
him  to  assess  and  to  apply  cons i s tend  y~Ms -own  utility  for  the- in-format  ion 
provided  by  alternative  sources. 
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The  idea  of  uti 1 i ty-based  aiding  of  information  selection  is  strongly 
supported  by  a study  of  McKendry,  Enderwick,  and  Harrison  (1971).  These 
investigators  were  interested  in  the  effects  on  decision-making  performance 
of  using  information  which  had  been  previously  judged  of  low,  medium,  and 
high  utility  to  the  task  by  the  decision  makers  themselves.  The  decision 
making  task  involved  the  tracking  of  a submarine  by  an  airborne  ASW  aircrew 
under  the  direction  of  a tactical  coordination  officer  (TACO)  who  had  at  his 
command  all  the  sensor  resources  of  the  real  aircraft. 


Subjective  information  value  (utility)  ratings  for  various  sensor 
returns  were  obtained  off-line,  by  direct  elicitation  techniques,  from  a 
total  of  39  experienced  TACOs.  Message  content  areas  were  judged  separately, 
and  the  total  subjective  information  value  of  a message  was  determined  by  a 
linear  combination  of  the  form: 


n 


where: 


b 

n 


= worth  of  information  in  a message 
= number  of  content  areas  in  a message 
= number  of  items  in  jth  content  area 
= average  utility  of  items  in  .jth  content  area 


(This  formulation  is  very  similar  to  the  one  derived  independently  by 
Perceptronics  investigators  for  t tie  mul ti -attribute  utility  of  information 
items;  see  Section  3.1.5.)  The  total  information  value  of  N messages  (B) 
was  obtained  by  summation  of  individual  message  ' b ' values. 


In  the  experiment,  aircrews  began  a tracking  exercise  with  a package 
of  information  items  totaling  low,  medium,  or  high  information  value.  The 


performance  measure  was  the  reduction  in  uncertainty  of  target  location; 
uncertainty  was  taken  as  proportional  to  the  ratio  of  remaining  search  area 
to  original  search  area.  Figure  2-1  shows  the  reduction  in  uncertainty  with 
time  when  working  with  low,  medium  and  high  value  information.  The  results 
indicated  that  performance  was  significantly  better  for  the  higher  information- 
value  conditions.  That  is,  crews  operating  with  the  high-utility  information 
were  able  to  reach  equivalent  uncertainty  levels  much  more  rapidly  than  crews 
operating  with  medium  or  low  utility  data.  In  fact,  the  time  improvement 
ratio  for  useful  reductions  of  uncertainty  lay  in  the  range  2.0  to  3.5. 

Thus  the  findings  of  this  highly  relevant  study  imply  that:  (1)  using 

information  of  higher  subjective  value  results  in  significantly  better 
decision-making  performance;  and  (2)  a linear  model  for  multi-attribute 
utility  appears  to  hold  up  well  over  various  combinations  of  tasks  and 
information  items,  that  is,  "individual  utilities  can  be  summed  to  yield 
aggregate  utilities  --  at  least  up  to  mixtures  of  three  things".  As  the 
authors  state,  the  findings  are  of  considerable  importance  to  builders  of 
decision-making  models. 

2 . 5 Information  Lit i 1 1 zation 

2.5.1  Information  Quantity  versus  Decision  Performance.  Does  more  information 
lead  to  better  decision  making?  The  pyschological  literature  provides 
several  instances  where  the  answer  is  "no".  For  example,  Schroeder  and 
Benbasat  (1975)  found  that  increases  in  the  level  of  information  detail  or 
in  the  historical  time  period  covered  by  the  information  did  not  affect 
decision  quality.  This  result  is  particularly  significant  since  the  subjects 
themselves  determined  how  much  information  they  would  receive. 

In  an  important  experiment,  Hayes  (1964)  studied  choice  decisions 
involving  which  of  several  airplanes  should  be  dispatched  to  investigate  a 
reported  submarine  sighting  in  a simulated  tactical  situation.  The 
alternative  planes  differed  with  respect  to  their  characteristics,  such  as 
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PERCENT  TARGET  LOCALIZATION  UNCERTAINTY 


LOW  UTILITY  INFORMATION 
MEDIUM  UTILITY  INFORMATION 
HIGH  UTILITY  INFORMATION 


ELAPSED  PROBLEM  TIME  IN  MINUTES 


FIGURE  2-1 

RESULTS  OF  AIRBORNE  ASW  PERFORMANCE  RUNS 
(McKendry,  Lnderwick,  and  Harrison,  1971) 
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pilot  quality,  speed,  radar  quality,  take-off  delay,  etc.  Each  characteristic 
could  take  on  one  of  several  "values"  which  could  be  ranked  from  best  to 
worst.  The  independent  variables  involved  the  number  of  planes  from  which  a 
choice  had  to  be  made  (4  or  8)  and  the  number  of  discriminating  characteristics 
(2,  4,  6,  or  8)  on  which  to  formulate  a choice.  Hayes  conducted  four 
experiments  in  all,  and  was  able  to  examine  the  effects  of  time  stress  and 
training. 

Hayes1  principal  findings  can  be  summarized  as  follows:  (1)  decision 

quality  was  superior  when  a choice  was  required  among  4 alternatives  rather 
than  among  8;  (2)  as  the  number  of  character! sties  (i.e.,  information 
attributes)  increased,  decision  time  increased  markedly  but  decision  quality 
did  not  improve;  that  is,  decisions  based  on  only  two  characteristics  were 
just  as  good  as  those  based  on  4,  6,  or  8 characteristics;  (3)  when  limited 
time  (10  seconds)  was  available  for  a decision,  increasing  the  number  of 
information  dimensions  could  actually  degrade  decision  quality;  (4)  attempts 
at  training  the  decision  makers  did  not  enable  them  to  learn  to  make  better 
decisions  with  increasing  numbers  of  information  dimensions.  The  important 
conclusion  to  be  drawn  from  this  research  is  that  it  is  easy  to  provide  a 
commander  with  more  information  than  he  can  assimilate  or  use  --  especially 
when  he  is  operating  under  time  pressure. 

It  is  interesting  to  note  that  Hayes'  result  concerning  time  stress  is 
consistent  with  a theory  recently  offered  by  Hogarth  (197!)).  Applying  to 
situations  where  a choice  must  be  made  among  a given  set  of  alternatives, 
the  theory  defines  task  complexity  and  cognitive  strain  as  increasing 
functions  of  both  the  number  of  characteristics  per  alternative  and  difficulty 
of  choice  between  alternatives.  Because  of  human  limitations  on  information 
processing,  optimal  decision  time  is  proposed  to  be  a concave  (i.e.,  inverted- 
U)  function  of  task  complexity.  The  same  idea  was  advanced  earlier  by 
Schroeder,  Driver,  and  Streufert  (1967),  who  support  the  theory  with 


empirical  data  which  repeatedly  demonstrate  an  inverted  U relationship 
between  environmental  complexity  (information  load)  and  level  of  information 
processing.  Thus,  it  is  possible  that  when  Hayes’  subjects  were  given  too 
many  characteristics  to  process  within  ten  seconds,  they  became  cognitively 
"overloaded”  and  could  not  perform  well. 


Huber,  O'Connell,  and  Cummings  (1975)  studied  the  effects  of  information 
specificity,  information  load,  and  group  structure  on  perceived  environmental 
uncertainty  in  an  exercise  using  the  Tactical  and  Negotiations  Game.  Examples 
of  "low  specificity"  and  "high  specificity"  messages  are  contrasted  below. 


High  Specificity  Message 

Two  carrier  based  F-2's  on  a routine 
patrol  from  NG-2A  sighted  a>id  attacked 
an  enemy  truck  convoy  in  sector  G-3. 
Five  trucks  were  completely  destroyed, 
two  partially  damaged. 


Low  Specificity  Message 

Carrier  based  airplanes  patrolling 
in  Western  Shamba  sighted  and 
attacked  an  enemy  truck  convoy. 
Damage  determined. 


The  "low  specificity"  message  can  be  considered  a reduced  information  set 
derived  from  the  "high  specificity"  message.  Group  decision  performance 
was  measured  in  terms  of  the  uncertainty  in  a vector  of  subjective 
probabilities  assigned  to  alternative  "enemy"  strategies  represented  in  the 
game  simulation.  No  significant  main  effect  was  obtained  for  information 
specificity,  suggesting  that  a decrease  in  message  detail  need  not  affect 
decision  uncertainty. 


Additional  evidence  that  reduction  of  information  detail  does  not 
necessarily  reduce  performance  levels  comes  from  a recent  study  by  Granda 
(1976)  which  investigated  whether  reduction  of  map  detail  reduces  the 
efficiency  of  human  information  gathering  and  tactical  decision  making  in 
a simulated  tactical  operations  system  (SJMTOS).  The  subjects  were  20  mid- 
level  Army  officers  that  performed  both  planning  and  combat  tasks  during  a 
SIMTOS  offensive  scenario.  One  group  performed  the  task  with  standard  Army 
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maps  while  the  other  group  were  only  allowed  access  to  maps  of  considerably 
reduced  detail.  Some  of  the  performance  measures  evaluated  were  combat 
effectiveness  scores,  area  captured,  amount  of  information  requested,  and 
information  processing  time.  No  significant  performance  differences  were 
found  between  users  of  the  reduced  detail  maps  and  the  standard  maps  despite 
the  fact  that  some  reduced-map  users  judged  them  to  be  inadequate. 

2.5.2  Utilization  Aiding  Through  User  Modeling.  A number  of  research  groups 
have  provided  empirical  support  for  the  advantages  of  user  modeling  in  aiding 
information  utilization.  In  a series  of  experiments,  Pask  and  Scott  (1971, 
1972)  demonstrated  that  when  information  presentation  techniques  are  matched 
with  the  information  processing  characteristics  of  the  user  (either  a 
"serialist"  or  a holist"),  cognitive  performance  is  enhanced  by  a ratio  of 
2.0  as  opposed  to  when  there  is  a mismatch.  Levit,  Heaton,  and  Alden  (1975) 
have  succeeded  in  categorizing  the  decision  styles  of  individuals  according 
to  three  bipolar  dimensions:  active/passive,  logical/intuitive,  and  abstract/ 

concrete.  In  a laboratory  experiment  using  a simulated,  automated  tactical 
environment,  these  researchers  provided  decision  aids  to  each  subject  in 
accordance  with  his  decision  style.  Although  the  results  were  not 
stati stical ly  conclusive  --  apparently  because  of  insensitivities  in  the 
system  performance  measures  --  the  soundness  of  individualized  decision 
support  in  computer-based  C3  systems  was  indicated. 

Investigators  at  Perceptror ics  have  developed  and  demonstrated  the 
technology  of  adaptive  utility  assessment  for  modeling  operators  (users)  in 
an  on-line  fashion.  Decision  support  systems  based  on  this  technology  have 
proved  successful  as  an  aid  to  information  acquisition  and  related  decision 
making  in  simulated  tactical  scenarios  (Weisbrod,  Davis  and  Freedy,  1977; 
Freedy  et  al.,  1976)  and  in  simulated  electronic  trouble  shooting  tasks 
(Freedy  and  Crooks,  1975).  Since  this  work  provides  a direct  background  for 
the  present  project,  its  contributions  and  results  are  reviewed  in  more 
detail  below. 
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2.5.3  ADDAM  Decision  Support  System.  Perceptronics ' initial  system  for 
decision  support,  termed  ADDAM,  was  developed  under  ARPA  and  ONR  funding. 
Adaptive  or  goal-directed  techniques  were  employed  extensively  in  ADDAM  for 
the  acquisition  of  operator  decision  strategies.  This  dynamic  modeling  is 
closely  related  to  the  "on-line  model  matching"  methods  practiced  in 
adaptive  manual  control  (Gilstad  and  Fu,  1970)  and  to  the  adaptive  linear 
models  used  to  augment  or  replace  the  expert  decision  maker  (Bowman,  1963; 
Kunreuther,  1969;  Dawes  and  Corrigan,  1974).  These  techniques  use  pattern 
recognition  or  learning  algorithms  to  estimate  behavioral  parameters.  The 
ensuing  models  are  then  used  to  train,  replace,  or  evaluate  the  operator. 
Perceptronics'  development  extends  this  field  of  work  by  placing  the  operator 
in  a real-time  interaction  with  his  model,  so  that  the  system  both 
descriptively  models  and  proscripti vely  aids  the  operator. 

The  ADDAM  decision  support  system  is  composed  of  a combination  of 
three  complementary  elements  --  a set  of  Bayesian  probability  aggregation 
programs,  a dynamic  model  for  tracking  operator  values  for  outcomes,  and  a 
strategy  recommendation  algorithm.  The  latter  two  elements  are  of  particular 
interest  here. 

Utility  Estimation.  In  ADDAM,  utility  estimation  is  realized  through 
the  use  of  a trainable  multi-category  pattern  classifier,  illustrated  in 
Figure  2-2.  As  the  operator  performs  the  decision  task,  this  on-line 
estimator  observes  the  operator's  choices  among  the  various  decision  options. 
The  estimator,  using  event  probabilities  as  inputs,  attempts  to  classify 
these  probability  patterns  by  adjusting  utility  weights  according  to  an 
adaptive  error  correcting  algorithm.  In  this  manner,  the  utility  estimator 
tracks  the  operator's  decision  making  and  learns  his  utilities.  Such  an 
approach  has  a number  of  advantages  compared  to  off-line  utility  estimation. 
Dynamic  estimation  observes  and  models  actual  behavior  rather  than  responses 
to  hypothetical  decisions.  It  does  not  interrupt  or  intrude  on  the  process 
of  decision  making.  And  it  responds  to  ongoing  changes  in  task  characteristics 


and  operator  needs.  The  characteristics  of  the  dynamic  utility  estimator 
have  been  evaluated  in  a variety  of  decision  contexts.  In  all  applications, 
the  estimates  of  multiple  dynamic  utilities  typically  converged  rapidly  to 
stable  and  distinct  values. 

Strategy  Recommendation.  This  element  of  the  ADDAM  system  follows 
naturally  from  the  probabi 1 ity  and  utility  estimators.  With  these  parameters 
defined.it  is  a simple  matter  to  recommend  individually  optimal  decisions. 

The  choice  with  the  greatest  expected  utility  is  determined  and  displayed  to 
the  operator.  The  recommendations  given  are  thus  based  on  the  operator's 
own  apparent  values,  and  are  organized  into  a normative  framework.  A certain 
generality  is  present  in  the  normative  processing,  since  the  recommendations 
are  not  restricted  to  the  identical  circumstances  used  for  training,  but  can 
be  applied  to  other  circumstances  of  the  same  structure. 

Over  the  past  three  years,  several  experimental  studies  have  been 
conducted  to  evaluate  the  decision  support  system  in  realistic  but  controlled 
circumstances.  In  one,  a fishing  fleet  simulation  task  was  used  as  the 
experimental  vehicle  (Weisbrod  et  al.,  1977).  The  task  involved  the  placement 
of  diverse  sensors  to  track  and  report  the  location  of  several  components  of 
a fishing  fleet  as  they  moved  over  an  ocean  expanse.  The  experiments  focused 
on  basic  system  validation  and  on  how  aiding  (sensor  recommendation)  affected 
the  "internal"  quality  of  decision  making.  The  experimental  evidence  studies 
indicated  that  (1)  the  adaptive  model  accurately  predicted  the  operator's 
decisions,  (2)  aiding  significantly  improved  decision  consistency,  (3)  aiding 
significantly  improved  decision  quality,  (A)  aiding  reduced  intersubject 
variability,  and  (5)  aiding  increased  decision  throughput. 

A recent  study  showed  that  external'  measures  of  decision  outcome, 
such  as  accuracy,  errors,  etc.,  are  also  improved  by  adaptive  aiding  (Freedy 
et  al.,  1976).  The  experimental  task  was  a new  ASW  version  of  the  fishing 
fleet  simulation.  Operators  tracked  the  movements  of  a submarine  and  an 
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interfering  object,  using  the  same  types  of  sensors  (sonobuoys,  helicopters, 
MAD's,  etc.)  available  in  Naval  ASW  exercises.  The  sensors  varied  in 
real iabi 1 i ty , specificity,  and  cost.  Unaided  operators  worked  alone,  using 
computer-generated  intelligence  reports.  Aided  operators  received  additional 
computer  assistance  in  the  form  of  (1)  Bayesian  sensor  output  evaluation, 
and  (2)  Utility-based  sensor  placement  recommendations. 

Results  showed  that  the  aided  group  performed  significantly  better 
than  the  control  group,  improving  their  mean  score  by  almost  a factor  of 
two  (88%).  Improvement  was  partially  attributed  to  a small  but  significant 
increase  in  the  number  of  decision  trials  completed  during  the  session. 

But  most  of  it  appeared  due  to  the  better  overall  quality  of  the  aided 
decisions.  That  is,  the  aided  operators  incurred  slightly  higher  costs, 
but  received  a much  greater  return  in  points,  and  a substantially  lower 
number  of  penalties.  Decision  consistency,  as  measured  by  mean  deviation 
from  maximum  expected  utility,  was  significantly  enhanced  for  the  aided 
group,  as  in  previous  studies.  Also,  in  replication  of  previous  studies, 
the  improved  performance  of  the  aided  group  was  accompanied  by  decreased 
intragroup  variability. 

In  most  man-machine  systems,  objective  performance  criteria  for  the 
immediate  task  are  not  well  defined,  or  are  only  indirectly  related  to  long 
term  system  goals.  This  indeterminacy  is  particularly  evident  in  systems 
operating  in  dynamic  environments,  where  the  results  of  earlier  decisions 
affect  later  decisions.  Such  systems  thus  rely  heavily  on  the  operator's 
subjective  evaluation  of  the  situation  at  hand,  and  the  decisions  should  be 
based  on  measurable  subjective  preferences  (utilities)  of  the  operator. 
Findings  to  date  indicate  that  when  these  utilities  are  incorporated  into 
an  aiding  system,  significant  improvements  in  performance  can  occur. 
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3.  MULTIATTRIBUTE  MODEL  FOR  INFORMATION  SELECTION 

3.1  Overview 

3.1.1  Concept.  Steeb  (1975)  suggested  the  use  of  an  adaptive  multiattribute 
model  as  a means  for  automatically  selecting  arid  distributing  information  in 

a generalized  system  for  command  and  control.  The  present  model  (or  model - 
based  selection  system)  is  based  on  that  suggestion.  Essentially,  the  model 
conceptualizes  information  as  a multi-dimensional  entity  which  can  be 
decomposed  into  a set  of  measurable  attributes.  The  model  computes  an 
aggregate  multiattribute  utility  (MAU)  as  a weighted  sum  of  each  information 
attribute  level  (A^)  multiplied  by  the  importance  or  utility  of  the 
attribute  (U^).  The  calculated  MAU  of  an  information  item  is  used  as  the 
selection  criterion. 

3.1.2  System  Organization.  Figure  3-1  shows  the  major  components  of 
the  selection  system  in  block  diagram  form.  Incoming  information  of 
potential  utility  to  the  recipient  is  parameter ized  in  terms  of  both 
immediate  situational  factors  (situation  mask  vector)  and  the  intrinsic 
characteristics  of  its  source  (source  characteristic  vector).  Together 
these  two  vectors  contribute  to  the  computation  of  the  attribute  levels 
associated  with  each  item  of  information.  The  "information  utility  calculator" 
uses  as  inputs  (1)  the  attribute  levels  of  the  incoming  information,  and 

(2)  a vector  of  "attribute  weights"  which  have  been  dynamically  estimated 
for  a given  operator  by  an  adaptive  model.  Rased  on  these  inputs,  the 
overall  multi-attribute  utility  of  each  information  iten  is  calculated. 
Information  is  then  rank  ordered  along  a scale  of  information  utility,  and 
a selection  mechanism  is  applied  to  determine  what  specific  information  is 
presented  to  the  operator. 

3.1.3  Application  Test  Bed.  The  model-based  selection  system  was 
implemented  and  tested  for  a simulated  ASW  intelligence-gathering  and 
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tracking  task.  This  task,  which  is  described  further  in  Section  3.2, 
requires  an  operator  to  select  among  a wide  variety  of  potential  information 
sources  (individual  sensor  deployments)  in  order  to  follow  the  movements 
of  a hostile  submarine  and  interfering  non-hostile  whale.  The  task 
itself  was  based  on  an  existing  computer  simulation,  used  previously  by 
Perceptronics  in  several  studies  of  adaptive  decision  aiding.  In  order  to 
adapt  it  to  the  present  case,  the  existing  simulation  was  modified 
considerably  at  the  conceptual  level,  and  concommitant  changes  were 
introduced  into  the  task  procedure.  Only  the  features  of  the  simulation 
that  are  germaine  to  the  present  program  are  described  in  Section  3.2.  For- 
a more  complete  description  of  the  original  simulation,  the  reader  is 
referred  to  Freedy,  Davis,  Steeb,  Samet,  and  Gardiner  (1976). 

3.1.4  Model  Function.  The  major  function  of  the  model  is  to  relieve  the 
operator  of  the  need  to  choose  among  available  information  sources.  That 
is,  to  automate  his  information  selection  task.  The  model  accomplishes  this 
by  integrating  two  different  types  of  knowledge.  First,  the  model  takes 
into  account  some  basic  features  of  the  environment  or  state  of  the  world; 
these  features  essentially  define  the  operator's  current  information  needs. 

In  the  ASW  simulation,  this  information  is  expressed  by  an  intelligence 
report,  and  takes  the  form  of  probability  data  about  the  location  and 
status  of  the  objects  being  tracked.  The  second  type  of  knowledge  involves 
a representation  of  the  operator's  own  utilities  for  particular  attributes 
of  information,  that  is,  his  individual  attribute  weight  vector.  Included 
is  the  recognition  that  his  preferences  may  depend  upon  the  specific 
information  requirements  of  the  situation.  With  respect  to  the  ASW  task, 
the  requirements  break  down  to  whether  the  operator  is  searching  for  a 
submarine,  a whale,  or  both,  in  a particular  location.  Obviously,  both 
types  of  knowledge  are  quite  general izable,  and  can  be  similarly  applied 
to  situations  outside  the  present  ASW  context. 


I 
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S 3.1.5  MAU  Calculation.  Calculation  of  the  multi-attribute  utility  for 

|;  information  is  central  to  the  workings  of  the  model.  The  MAU  calculation 

is  shown  in  Figure  3-2  as  a two-step  process.  In  Step  1,  the  cross-product 

of  the  situation  mask  vector  and  the  source  characteristic  vector  results 
in  the  attribute  level  vector.  This  calculation  is  described  in  Section  3.3. 
Definitions  of  the  vectors  themselves  are  provided  below. 

Situation  Mask  Vector.  The  purpose  of  this  vector  is  to  define, 
for  a specific  situation,  the  general  information  needs  of  the  information 
recipient.  The  parameters  reflect  both  the  local  environment  and  the 
task  objective.  For  example,  in  the  ASW  tracking  simulation,  the  content 
of  the  situation  mask  vector  answers  such  questions  as:  Is  the  object 

of  search  a submarine,  a whale,  or  both?  If  a submarine  is  being  sought, 
is  it  necessary  to  discriminate  between  whether  it  is  in  a resting  or  floating 
state?  etc. 


Source  Characteristic  Vector.  This  vector  describes  the  characteristics 
of  a particular  information  source.  These  characteristics  are  properties  which 
are  assumed  to  affect  the  decisions  of  an  operator  when  he  selects  among 
information  items,  or  messages.  For  example,  how  much  does  the  information 
cost?  To  what  does  it  pertain?  How  reliable  is  it?  In  the  case  of  the 
ASW  simulation,  for  example,  can  the  sensor  discriminate  between  the  presence 
of  a submarine  and  a whale,  or  between  a floating  and  resting  submarine? 


In  Step  2 of  the  MAU  calculation,  the  dot-product  of  the  attribute 
level  vector  and  the  attribute  weight  vector  provides  the  aggregate  MAU 
value.  Derivation  of  the  attribute  weights,  or  utilities,  is  described 
further  in  Section  3.4.  In  essence,  the  multiattribute  representation  of 
the  operator's  information  preferences  is  built  up  during  performance 
sessions  in  which  the  operator  must  make  a series  of  choices  among  potential 
information  sources  in  response  to  varying  situational  needs.  Although 
the  ASW  task  actually  requires  him  to  choose  amnq  sensors  tor  deployment. 


3-4 


STEP  1: 


Situation 

Mask 

Vector 


X 


Source 

Characteristic 

Vector 


Attribute 

Level 

Vector 


V 

1 

1 

M2 

C2 

A2 

M3 

C3 

A3 

M4 

X 

C4 

= 

A4 

M5 

C5 

A5 

M6 

C6 

A6 

M7 

C7 

A7 

_ — 

_ _ 

_ _ 

STEP  2:  Attribute 

Level 
Vector 


Attribute 

Weight  = MAU 

Vector 


1 

J 

U1 

A 2 

u2 

A3 

U3 

A4 

, 

U4 

A5 

U5 

A6 

U6 

A7 

U7 

„ — 

_ 

I 


FIGURE  3-2 

CALCULATION  OF  INFORMATION  UTILITY 


PERCEPTRONICS 


each  deployed  sensor  can  be  viewed  as  a message,  or  as  a mol t i-attr i bute 
item  of  information,  and  each  choice  as  a selection  among  all  alternative 
messages  that  could  be  generated  and  presented. 


E 


As  the  operator  proceeds,  an  adaptive  model  performs  in  parallel 
with  him,  by  predicting  which  information  sources  he  will  select.  The  model 
"learns"  through  an  error  correction  procedure  which  dynamically  adjusts 

‘ 

estimates  of  the  weights  that  the  operator  places  on  the  relevant  information 
attributes,  and  the  predictions  converge  upon  the  actual  information 
selection  behavior  of  the  operator.  At  this  point,  the  model,  which  now 
accurately  mimics  the  operator's  information  selec' ' oreferences,  can 
take  over  the  function  of  making  the  selection  amo.  lable  information. 

Thus,  the  system  assumes  a mode  in  which  information  i^  utomatically 
selected  for  presentation  to  the  operator,  in  accord  with  his  own  generalized 
preferences.  The  selection  rules  themselves  can  also  be  tailored  to  the 
situational  needs  and  the  operator's  characteri sties  or  desires.  Those 
considered  in  the  present  study  are  discussed  in  Section  3.5. 


3 . 2 ASW  Si mulati o n 

The  experimental  simulation,  although  relatively  primitive,  includes 
the  salient  decision  features  of  ASW  localization  and  tracking.  The 
simulation  centers  on  an  aircraft  carrier,  proceeding  with  a normal  complement 
of  ASW  resources.  It  is  assumed  that  the  simulated  ASW  task  begins  after 
a hostile  submarine  and  an  interfering  whale  have  been  detected,  and 
fixed-wing  aircraft  have  deployed  sono-buoys  over  the  entire  potential 
attack  zone.  The  attack  zone,  displayed  on  a graphics  terminal,  is  assumed 
to  ranain  stationary  with  respect  to  the  aircraft  carrier.  Figure  3-3 
illustrates  the  configuration. 

3.2.1  Operator's  Task.  The  ASW  simulation  involves  a single  operator. 

His  task  is  to  lllocate  sensor  resources  in  order  to  track  and  report  on 
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CONFIGURATION  OF  ASW  SIMULATION 


PERCLPTRONICS 


the  subsequent  movements  of  the  submarine  and  the  whale  in  the  well-defined 
ocean  expanse.  The  basic  cycle  is  illustrated  in  Figure  3-4.  It  begins 
with  the  deployment  of  sensors  which  report  back  what  they  have  detected. 

The  operator  interprets  these  results  and  reports  the  status  of  the  tracked 
objects.  At  this  point  in  the  performance  of  the  task,  the  operator  receives 
an  intelligence  report  which  aids  him  in  the  placement  of  new  sensors,  and 
so  starts  another  task  cycle.  All  previously  deployed  sensors  are  removed 
at  the  end  of  each  status  reporting  cycle. 

3.2.2  Sensor  Characteristics.  To  monitor  the  movements  of  the  objects, 
the  operator  deploys  sensors  which  differ  with  respect  to  level  of  detection, 
reliability,  and  cost.  The  scenario  allows  the  operator  to  deploy  one 

of  five  types  of  sensors  at  each  location;  Table  3-1  lists  the  sensors  and 
summarizes  their  properties.  By  evaluating  the  intelligence  report  obtained 
from  the  system  as  a result  of  his  last  report  on  the  objects'  location, 
the  operator  decides  on  the  sensor-location  combinations  which  will  provide 
him  with  the  information  he  needs  to  continue  to  track  the  objects. 

After  deploying  the  sensors  the  operator  receives  information 
about  their  output.  An  ' H ' sensor,  for  example,  can  have  one  of  two  possible 
outcomes;  "positive",  indicating  presence  of  a floating  submarine  in  the 
sector,  and  "negative",  'ndicating  the  absence  of  a submarine.  Since  sensors 
are  not  perfectly  reliable,  their  response  may  be  erroneous.  The  reliability 
(r)  of  the  sensor  is  the  probability  that  it  will  give  a true  report.  The 
complement  of  reliability  (1-r)  is  the  likelihood  that  the  sensor  will  give 
a false  report.  A false  report  may  he  one  of  two  types,  either  failing 
to  report  a detectable  object  when  it  is  actually  there  (false  negative) 
or  reporting  the  presence  of  the  object  when  it  is  not  actually  there  (false 
positive) . 

3.2.3  Status  and  Intelligence  Reports.  After  eiving  tl  ensor 

outputs,  the  operator  is  required  to  make  a status  report  reflecting  his 


TABLE  3-1 


PROPERTIES  OF  ASW  INFORMATION  SOURCES 


SENSOR  TYPE 

LEVEL  OF  DETECTION 

RELIABILITY 

COST 

(B) 

Object* 

.60 

1 

(Ml) 

Submarine  Floating 
Submarine  Resting 

.95 

10 

(M2) 

Submarine  Floating 
Submarine  Resting 

.80 

5 

(H) 

Submarine  Floating 

.90 

7 

(D) 

Submarine  Floating 
Submarine  Resting 
Whale 

.70 

4 

A Sono-Buoy  reports  only 
identify  whether  it  is  a 


that  an  object  has 
whale  or  submarine 


been  detected  but  does  not 


I 


best  estimate  of  the  present  location  of  both  the  submarine  and  the  whale. 
He  bases  his  decisions  on  the  prior  probabilities  of  object  location  as 
determined  by  the  intelligence  report,  and  on  the  sensor  outputs  in  light 
of  the  known  error  rates  of  the  sensors  deployed.  Essentially,  he  must 
arrive  at  a posterior  estimate  by  integrating  the  newly  obtained  sensor 
data  with  the  prior  data.  Because  the  status  decisions  affect  the 
subsequent  intelligence  report,  the  sensor  deployment  decisions  are,  in 
turn,  dynamically  influenced  by  the  status  decisions. 

The  intelligence  report  which  aids  the  operator  in  deploying 
sensors,  is  derived  from  the  operator's  report  on  the  status  of  the  objects 
being  tracked  and  from  expert  assessments  of  the  behavior  of  these  types 
of  objects.  The  system  assumes  that  the  operator  has  correctly  reported 
the  location  and  heading  of  each  object.  By  aggregating  the  conditional 
probabilities  of  state  transformations  (elicited  from  experts),  it  makes 
a Bayesian  estimate  of  their  next  location.  Thus,  the  intelligence 
report  contains  the  probabilities  that  each  object  will  be  in  each  sector 
of  the  attack  zone.  The  whale  may  move  to  an  adjacent  location  or  remain 
in  the  same  location.  The  submarine  may  move  likewise  or  remain  in  the 
same  location,  either  floating  in  the  ocean  or  resting  on  the  bottom. 

3 . 3 Attribute  Level  Calculation 

3.3.1  Information  Attributes.  A set  of  s^ven  attributes  was  required 

to  model  successfully  the  information  preference^  of  an  operator  performing 

\ . 

the  simulated  ASW  task.  These  attributes  were  arrived  at  through  an 
evolut ionary,  model-development  process.  Performance  data  and  experience 
with  the  ASW  simulation  (Freedy,  et  al , 1976)  provided  an  initial  set 
of  attributes  for  implementation.  The  flexibility  of  the  model  allowed 
this  set  to  be  iteratively  tested  for  predictive  sensitivity  and  to 
consequently  be  refined.  The  refinement  procedure  involved  both  the 
modification  of  existing  attributes  and  the  replacement  of  insensitive 
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ones  with  more  promising  ones.  The  development  process  was  continued 
until  a distinct  set  of  attributes  would  satisfactorily  predict  information- 
selection  behavior  for  different  operator  preference-strategies.  Aspects 
of  this  process  are  illustrated  in  Chapter  4.  Table  3-2  lists  the  final 
set  of  attributes. 

TABLE  3-2.  INFORMATION  ATTRIBUTES  FOR  ASW  TASK 

(1)  Cost  of  Submarine  Information 

(2)  Cost  of  Whale  Information 

(3)  Expected  Submarine  Information  Content 

, (4)  Expected  Whale  Information  Content 

(5)  Submarine  Information  Parsimony 

(6)  Submarine/Whale  Discriminabil ity 

(7)  Submarine  Status  Discriminabil ity 

3.3.2  AttrjiJxit_e  Level  Vector.  The  level  of  each  attribute  for  a specific 
sensor  source  is  determined  by  the  multiplication  of  entries  in  the 
situation  mask  vector  and  the  source  character istic  vector.  Table  3-3 
defines  the  situation  mask  and  source  characteristic  entries  for  each  of 
the  ASW  attributes.  The  situation  mask  and  source  characteristic  vectors 
are  scaled  so  that  each  attribute  level  ranges  from  0 to  1 . Further,  the 
orientation  is  arranged  such  that  each  att  ihute  contributes  positively 

to  the  overall  aggregate  MAU.  That  is,  holding  all  other  attribute  levels 
constant,  an  increase  in  any  attribute  level  increases  the  MAU.  The 
individual  entries  should  become  more  clear  in  the  description  of 
attributes  given  below. 

3.3.3  Attribute  Descriptions.  Following  are  more  detailed  descriptions 
of  the  information  attributes  for  the  ASW  simulation.  They  exemplify  the 
use  of  the  situation  mask  and  the  source  characteristic  to  model 
information  in  a specific  selection  situation. 
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TABLE  3-3 


DEFINITION  OF  MASK  AND  INFORMATION 
CHARACTERISTIC  FOR  EACH  ATTRIBUTE 


Situation 

Mask 


1 when  Ps>0, 
0 otherwise 


Information 

Characteristic 


2 

1 when  Pw>0, 
0 otherwise 

Cost 

3 

PS 

Rel iabi 1 ity 

4 

pw 

Rel iabil ity 

5 

1 when  Pr=0 
and  P.j=0, 

0 otherwi se 

1 when  both  F and  R 
can  be  detected 
0 otherwise 

6 

HS,W 

1 when  S and  W can 
be  discriminated, 

0 otherwise 

7 

hf,r 

1 when  F and  R can 
be  discriminated, 

0 otherwise 

: P 

= Probability 

S 

= Submarine  (either  floatiny 

or  resting) 

F 

= Submarine  Floating 

R 

= Submarine  Resting 

= Whale 


= -(P,.  1 og ^ Ps  + Pw  log^  P^),  with  prohabil  ities  normal  i?< 


*(Pp  1 og ^ Pp  + P,  log,,  P^),  with  probabilities  normalized 


I 
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(1)  Cost  of  Submarine  Information.  The  cost  of  submarine 
information  is  the  expenditure  required  to  receive  information 
from  a sensor  about  the  presence  of  a submarine.  By  virtue 

of  the  situation  mask,  the  attribute  comes  into  play  (i.e., 
is  non-zero)  only  when  the  submarine  is  being  searched  for 
(i.e.,  the  probability  of  submarine  presence  is  greater  than 
zero).  The  source  characteristic  of  cost  used  in  the 
computation  of  the  attribute  is  do  ived  from  the  sensor 
properties  listed  in  Table  3-1.  The  formula  used  to  convert 
the  absolute  value  of  cost  (c)  into  an  appropriate  relative 
quantity  necessary  for  the  attribute-level  computation  was 
(11  -c )/ 1 0.  This  transformation  s necessary  to  provide 
cost  with  a 0 to  1 range  and  to  conform  to  the  orientation 
standard  of  the  model.  Thus,  a decrease  in  absolute  cost 
(Table  3-1)  leads  to  an  increase  n the  source  characteristic 
of  cost  which,  assuming  everyti inq  else  equal,  increases  the 
MALI  of  the  information.  More  ip; 1 y , a decrease  in  information 
cost  increases  information  utility. 

(2)  Cost  of  Whale  Information.  The  cost  of  whale  information  is 
the  expenditure  required  to  obtain  information  from  a sensor 
about  the  presence  of  a whale,  'h  situation  mask  activates 
the  attribute  only  when  the  whale  s being  sought  (i.e.,  the 
probability  of  whale  presence  ■ ■. eeds  zero).  The  source 
characteristic  of  cost  is  set  and  interpreted  in  the  same  way 
as  for  the  cost  of  submarine  information.  Thus,  the  only 
difference  between  the  first  two  attributes  is  that  one  is 
relevant  when  the  submarine  is  being  tracked  and  the  other 

is  relevant  witen  the  whale  is  being  tracked.  Of  course,  both 
are  activated  when  both  objects  are  searched  for  within  the 
same  location. 
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(3)  Expected  Submarine  Information  Content.  This  attribute  is 
the  product  of  the  prior  probability  that  the  submarine  is 
present  (situation  mask,  P<.,  determined  by  the  intelligence 
report)  and  the  source  characteristic  value  corresponding 

to  the  information  reliability.  The  latter  value  is  derived 
from  the  sensor  properties  (Table  3-1);  as  in  the  case  of 
the  cost  attributes,  a transformation  was  required  to  spread 
reliability  (r)  over  a 0 to  1 scale.  The  formula  used  was 
(R  - .59)/. 36.  The  attribute  is  proportional  to  the  likelihood 
of  submarine  detection  (i.e.,  x r);  when  P<.  = 0 within  the 
location,  there  is  no  likelihood  of  finding  a submarine  there 
and  the  attribute  does  not  contribute  to  the  MAU  calculation. 

An  increase  in  P<-  and/or  r increases  the  chances  of  a true 
positive  report  of  a submarine,  thus  increasing  the  utility 
of  the  information. 

(4)  Expected  Whale  Information  Content.  This  attribute  is  the 

product  of  the  prior  probability  that  the  whale  is  present 

(situation  mask,  P^,  determined  by  the  intelligence  report) 

and  the  source  characteristic  value  corresponding  to  the 

information  reliability.  The  source  characteristic  of 

reliability  is  set  and  interpreted  in  the  same  way  as  for 

expected  submarine  information  content.  The  attribute  is 

proportional  to  the  likelihood  of  whale  detection  (i.e.,  Pw  x r) 

when  P. , = 0 within  the  sector,  there  is  no  likelihood  of 
w 

finding  a whale  there  and  the  attribute  does  not  contribute 

to  the  MAU  calculation.  An  increase  in  P.,  and/or  r increases 

w 

the  chances  of  a true  positive  report  of  a whale,  thus 
increasing  the  utility  of  the  information.  As  in  the  case 
of  the  cost  attributes,  the  two  expected  information  content 
attributes  differ  only  with  respect  to  the  object  of  search. 
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(5)  Submarine  Information  Parsimony.  The  attribute  of  parsimony 

reflects  the  use  of  a sensor  whose  capability  matches  the 

information  requirements  on  hand.  In  the  present  case, 

parsimony  refers  to  the  deployment  of  a sensor  which  can 

detect  only  the  precise  object  of  search,  i.e.,  a floating 

submarine.'  The  situation  mask  activates  the  attribute  only 

when  a floating  submarine  is  the  singular  target  being  sought 

(i.e.,  the  probabilities  of  a resting  submarine  (PR)  and  a 

whale  (P^)  are  both  zero).  The  source  character istic  is  set 

to  1 when  both  a floating  and  resting  submarine  can  be  detected 

2 

and  is  set  to  0 otherwise.  Only  the  H sensor  is  parsimonious 
with  respect  to  the  detection  of  a floating  submarine  since 
it  cannot  detec+  anything  else.  Ideally,  an  increase  in 
information  parsimony  should  increase  information  utility. 


(6)  Submar  ine/Wha  1 e Pi  scrim  inai. , i ity.  This  attribute  concerns 

the  discrimination  between  a submarine  and  a whale.  Discriminabil ity 
reflects  whether  the  information  source  can  precisely  identify 
a detected  object.  The  property  of  discriminabil ity  is  different 
from  that  of  level  of  detection  (Table  3-1).  For  example, 
the  B sensor  can  detect  both  the  submarine  and  the  whale  but 
it  cannot  discriminate  between  them.  In  fact,  only  the  D sensor 
can  discriminate  between  the  two  objects;  thus  it  is  assigned 
a 1 in  the  source  character ist  ic  vector  whereas  the  other  sensors 


There  was  no  need  for  a parsimony  attribute  with  respect  to  a resting 
submarine  because  the  characteristics  of  the  ASW  scenario  were  such  that 
whenever  there  was  some  probability  of  a resting  submarine  there  was  also 
some  probability  of  a floating  submarine.  That  is,  the  very  possibility 
of  a resting  submarine  presence  required  discriminatioi  etween  the  two 
submarine  states. 


"Because  of  a programming  error  caught  too  late,  the  orientation  of  parsimony 
was  reversed  from  what,  was  originally  intendel  Thus,  as  modeled,  when  the 
attribute  level  for  parsimony  equaled  1,  the  information  was  actually  the 
opposite  of  parsimonious.  However,  alfhough  this  n ror  makes  the  orientation 
of  parsomony  inconsistent  with  tiiat.  of  the  ther  att  ibut.es,  it  does  not 
affect  the  performance  of  the  MA  model  in  anv  .ubstant ive  way. 


are  assigned  a 0.  The  situation  inask  contains  the  relative 
uncertainty  associated  with  the  simultaneous  presence  of 
the  submarine  and  whale.  In  effect,  the  mask  modulates  the 
attribute  level  with  respect  to  the  need  of  the  operator  to 
discriminate  between  the  two  objects  in  the  local  environment; 
the  more  equal  the  prior  probabilities  of  the  respective 
objects  being  found  in  the  same  location,  the  higher  the 
uncertainty  level.  As  both  the  need  to  discriminate  and  the 
capability  to  do  so  increase  so  does  information  utility. 

( 7 ) Submarine  Floating/Submarine  Resting  Discriminabil ity . This 
attribute  concerns  the  discrimination  between  the  states  the 
submarine  can  assume  --  either  floating  or  resting.  The  Ml, 
M2,  and  D sensors  can  make  this  discrimination  but  the  H and 
B sensors  cannot;  the  capability  is  reflected  by  a 1 or  0, 
respectively,  in  the  source  characteristic  vector.  The 
mechanics  of  the  computation  of  this  attribute  and  the  impact 
of  the  components  on  information  utility  are  identical  to 
those  for  submarine/whale  discriminabil ity. 


3.4 


Attri bute 


Estimates  for  an  operator's  attribute  weights,  or  his  utility  for 
that  attribute,  are  provided  by  the  adaptive  portion  of  the  model.  The 
weights  are  learned  (or  trained)  during  sessions  where  an  operator  performs 
the  ASW  tracking  task  by  choosing  freely  among  the  five  possible  sensors 
for  deployment  in  specific  locations  of  interest.  The  model  begins  with 
equal  weights  assigned  to  each  attribute  and  then  dynamically  adjusts 
them  in  accordance  with  a simple  training  rule. 

3.4.1  Utility  Estimator.  The  dynamic  utility  estimation  technique  is 
based  on  a trainable,  multi-category  pattern  classifier.  Figure  3-b 
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illustrates  the  mechanism.  As  the  operator  performs  the  task,  the  on-line 
utility  estimator  observes  his  choices  among  the  available  information 
sources  (messages)  at  each  location,  and  views  his  decision-making  as 
a process  of  classifying  patterns  of  information  attributes.  The  utility 
estimator  attempts  to  classify  the  attribute  patterns  by  means  of  a linear 
evaluation  (discriminant)  function.  These  classifications  are  compared 
with  the  operator's  choices.  Whenever  they  are  incorrect,  an  adaptive 
error-correction  training  algorithm  is  used  to  adjust  the  utilities.  A 
comprehensive  discussion  of  this  technique  can  be  found  in  Freedy,  et  a 1 . ( 1 976 ) . 

3-4.2  Training  Algorithm.  On  each  trial,  the  model  uses  the  previous 
utility  weights  (IK)  for  each  attribute  (j)  to  compute  the  multi-attribute 
utilities  (MAIL)  for  each  sensor  (i)  in  each  plausible  location  of  ocean 
( i . e. , board  square) : 

mu i = jE,  Aji  uj 

For  all  squares  where  there  is  a non-zero  probability  of  a submarine 
or  whale,  the  model  predicts  that  the  operator  will  always  prefer  the 
information  source  with  the  maximum  MAU  value.  If  the  prediction  is  correct 
(i.e.,  the  operator  chooses  the  sensor  with  the  highest  MAU),  no  adjustments 
are  made  to  the  utility  weights.  However,  if  the  operator  chooses  a sensor 
having  a MAU  less  than  that  of  the  predicted  sensor,  the  model  then  adjusts 
the  utility  weights  by  pairing  the  chosen  sensor  with  the  predicted  sensor 
and  applying  the  error  correction  training  algorithm.  In  this  manner, 
the  utility  estimator  "tracks"  the  operator's  information  selection  behavior 
and  learns  his  utilities  or  weights  for  information  attributes.  The  training 
rule  used  to  adjust  the  weights  associated  with  each  of  the  attributes  is 
illustrated  in  Table  3-4. 


TABLE  3-4 


WEIGHT-TRAINING  RULE* 


Adjusted 

Weight 

<V 

Previous 

Weight 

'V 

Adjustment 

Factor 

(A) 

CORRECTION 

Chosen 

Information-Source 
Attribute  Level 

(AJC) 

DIFFERENCE 

Predicted 

Information-Source 
Attribute  Level 

^AjP^ 

U1  = 

U1  + 

A 

<Alc 

V 

U2  = 

U2  + 

A 

(a2c 

A-  ) 

U3  = 

U3  + 

A 

^A3c 

A0  ) 
3p' 

U4  = 

U4  + 

A 

<A4c 

fl4p> 

U5  = 

U5  + 

A 

<A5c 

%> 

U6  = 

U6  + 

A 

<A6c 

A6P> 

U7 

U7  + 

A 

(A7c 

A7P> 

* was  set  equal  to  .4 
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Information  Selection 


3.5.1  One  Message  Per  Location.  The  final  component  of  the  multiattribute 
model  involves  the  criteria  for  selecting  information.  The  model  as  applied 
to  the  ASW  simulation  allowed  for  two  types  of  selection  processes.  The 
first  process  was  confined  to  the  selection  of  a single  item  of  information 
(i.e.,  sensor)  for  each  grid  element  having  a non-zero  probability  that 
either  a submarine  or  whale  would  be  present.  By  computing  the  MAU  for 
each  of  the  five  possible  sensors  which  could  provide  information  on  objects 
in  a chosen  element,  the  sensor  with  the  highest  utility  is  selected  as 
providing  the  most  preferred  information  for  that  element. 

3.5.2  Message  Pruning.  The  second  selection  process  begins  with  the 
output  of  the  first  process.  Given  a set  of  messages  representing  information 
about  locations  where  the  objects  might  be  found,  utility-based  criteria  can 
be  applied  to  select,  for  presentation  only,  the  best  messages  among  them. 

That  is,  once  all  useful  messages  have  been  ranked  in  order  of  decreasing 
utility,  any  number  of  selection  algorithms  or  pruning  rules  can  be  applied. 
For  example,  a simple  rule  is  to  transmit  some  fixed  number  of  the  highest 
utility  messages.  Thus  the  operator  would  always  receive  four  messages  per 
cycle,  or  five,  etc.  Another  possibility  is  to  transmit  only  those  messages 
whose  MAU  exceeds  a certain  threshold  value.  A more  sophisticated  rule  is 

to  successively  transmit  each  next  ranked  message  only  if  its  utility 
exceeds  some  fixed  percentage  of  the  total  utility  of  messages  already 
transmitted.  This  rule  results  in  a varying  number  of  messages  being 
selected,  depending  on  operator  preferences  and  the  immediate  situation. 

Within  a given  strategy  (i.e.,  fixed  set  of  attribute  weights),  the  prior 
probabilities  will  influence  the  characteristics  of  the  cumulative  utility 
curve,  and  thus  the  number  of  sensors  selected  before  the  pruning  cutoff 
point  is  reached.  The  latter  pruning  rule  was  implemented  and  evaluated 
in  the  present  study,  using  15  as  the  incremental  criterion. 


4.  DEMONSTRATION  AND  EVALUATION 


Because  of  the  prototypical  nature  of  the  multi-attribute  information 
utility  model  developed  here,  an  exploratory  research  approach  was  taken  to 
evaluate  and  demonstrate  its  properties  and  capabilities.  This  chapter 
reviews  the  main  stages  of  the  approach.  These  include  certain  aspects  of 
the  model -development  process,  a structured  study  of  model  dynamics  and 
performance,  and  a systematic  empirical  demonstration  of  how  the  model 
contributes  to  human  task  performance.  The  experimental  vehicle  used 
throughout  the  evaluation  and  demonstration  was  the  simulated  ASW  tracking 
task. 

4 . 2 Differentiation  of  Attributes 

4.2.1  Situation  Specificity.  As  mentioned  in  Section  3.3.1,  an  iterative 
developmental  process  was  employed  to  determine  the  set  of  information 
attributes  for  use  in  the  systematic  evaluation  of  the  model's  characteristics 
and  capabilities.  During  this  process,  much  was  learned  about  the  design 
requirements  of  a model  intended  to  replace  the  information  selection 
function  of  a human  operator.  In  particular,  it  became  immediately  apparent 
that  there  was  a need  for  the  model  to  discriminate  between  the  operator's 
preferences  in  one  environmental  situation  as  opposed  to  his  preferences  in 
another.  The  demonstration  of  this  requirement  is  worthwhile  because  it  led 
to  the  development  of  the  general  concept  of  the  situation  mask. 


4.2.2  Effect  on  Convergence.  The  seven  attributes  used  to  model  the 
operator's  preferences  (Table  3-2)  include  a pair  of  cost  attributes  and  a 
pair  of  expected  information  content  attributes.  Each  pair  of  attributes, 
whether  for  cost  or  for  expected  information  content,  is  identical  with 
respect  to  the  information  source  characteristic.  They  differ,  however. 
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with  respect  to  the  situation  mask,  which  reflects  whether  the  object  of 
search  is  a submarine  or  a whale.  The  obvious  questions  is  why  one  cost 
attribute  and  one  expected  information  content  attribute  doesn't  suffice? 

Why  are  two  separate  attributes  required  in  each  case? 

The  answer  to  this  important  question  can  be  simply  illustrated. 

Suppose  that  the  operator  always  preferred  an  expensive,  high  expected 
information  content  source  when  searching  for  a submarine,  but  that  he 
always  preferred  a cheaper,  lower  expected  information  content  source  when 
searching  for  a whale.  If  a single,  undifferentiated  cost  or  expected 
information  content  attribute  were  implemented,  the  adaptive  model  would 
interpret  the  operator's  behavior  as  inconsistent,  and  would  repeatedly 
adjust  the  weight  in  opposing  directions,  resulting  in  an  oscillating, 
nonconvergent  weight-training  curve,  as  shown  in  Figure  4-1.  The  operator 
is  actually  quite  consistent,  but  only  with  respect  to  each  search  situation. 
When  the  attribute  is  differentiated  into  two  situation-specific  attributes, 
the  attribute  weights  can  be  trained  to  convergent  values  as  illustrated  in 
Figure  4-2. 

4 . 3 Adaptation  of  Attribute  Weights 

4.3.1  Definition  of  Selection  Strategies.  The  next  stage  of  evaluation  was 
to  demonstrate  the  general  capability  of  the  model  to  adapt  to  an  individual 
operator's  information  selection  strategy  to  the  point  of  correctly  predicting 
his  information  preferences.  To  accomplish  this  systematical ly , we  identified 
logical  information  selection  strategies  which  could  be  employed  by  an 
operator  while  he  performed  the  ASW  tracking  task.  These  strategies  are 
summarized  in  Table  4-1.  Essentially  the  strategies  differ  with  respect  to 
the  way  the  operator  treats  the  information  attributes  relating  to  cost, 
information  content,  and  parsimony.  Within  each  strategy,  it  was  assumed 
that  the  operator  would  always  want  to  discriminate  a submarine  from  a whale 
and  a floating  submarine  from  a resting  submarine  whenever  the  task  situation 
required  him  to  do  so. 
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TABLE  4-1 


DEFINITION  OF  INFORMATION  PREFERENCES 
FOR  SELECTION  STRATEGIES  TESTED 

Strategy  Submarine  Whale 


Cost  & Expected  Information  Content  Medium  Low 

Parsimony  Yes 

Submarine-Whale  Discrimination  Yes  

Submarine  Status  Discrimination  Yes 


Cost  & Expected  Information  Content  High  High 

Parsimony  No 

Submarine-Whale  Discrimination  Yes  

Submarine  Status  Discrimination  Yes 


Cost  & Expected  Information  Content  Low  Low 

Parsimony  Yes 

Submarine-Whale  Discrimination  Yes  

Submarine  Status  Discrimination  Yes 


Cost  & Expected  Information  Content  Low  High 

Parsimony  Yes 

^ Submarine-Whale  Discrimination  Yes  

Submarine  Status  Discrimination  Yes 


4.3.2  Model  Performance  with  Consistent  Strategy.  The  predictive 
performance  of  the  model  with  respect  to  each  strategy  was  first  tested 
under  ideal  conditions.  That  is,  each  strategy  was  carried  out  with  perfect 
consistency  throughout  a sequence  of  tracking  trials.  During  this  sequence, 
which  involved  24  trials,  an  expert  operator  performed  the  tracking  task  in 
a manual  information-selection  mode.  A trial  consisted  of  a set  of 
information-selection  decisions,  each  decision  being  associated  with  a 
particular  location  of  search  (i.e.,  the  grid  element  of  Figure  3-1).  Based 
i the  intelligence  report  of  the  probabilities  of  object  presence  in 
several  locations,  the  operator  selected,  for  each  location,  the  one 
information  source  (among  the  five  available)  from  which  he  most  preferred 
to  obtain  output.  For  example,  when  searching  for  the  submarine  by  itself 
under  Strategy  II,  the  operator  would  always  prefer  an  Ml  sensor  because  it 
had  the  highest  cost,  gave  the  highest  expected  information  content,  was 
not  parsimonous,  and  could  --  if  necessary  --  discriminate  between  a floating 
and  resting  submarine.  Thus  an  operator's  strategy  decomposes  into  a set 
of  consistent  preferences  which  are  associated  with  the  relevant  properties 
of  potential  information  sources. 

The  model  succeeded  in  adapting,  i.e.,  arriving  at  correct  predictions 
of  operator  information-selection  choices,  for  each  of  the  four  distinct 
strategies.  At  the  beginning  of  each  learning  session,  each  of  the  seven 
attributes  were  set  to  an  arbitrary  value  of  unity.  As  described  in  Section 
3.4,  the  model  immediately  began  to  perform  in  parallel  with  the  operator. 

The  MALI  was  computed  for  each  information  source  on  the  basis  of  the  current 
attribute  weights,  and  the  model  predicted  that  the  operator  would  prefer 
the  source  with  the  highest  MAU.  If  he  did  not,  the  error  correction 
procedure  was  applied  to  adjust,  the  weights.  Since  the  course  of  adaptation 
of  the  attribute  weights  was  similar  for  each  of  the  four  strategies 
evaluated,  the  typical  adaptation  pattern  will  be  illustrated  for  only  one 
of  them,  namely  Strategy  II.  The  adaptation  of  the  weights  for  this  strategy 
is  depicted  in  Figure  4-3. 
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FIGURE  4-3 

ADAPTION  OF  ATTRIBUTE  WEIGHTS 
FOR  STRATEGY  II  (Ml,  D) 
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As  illustrated,  the  weights  for  only  five  of  the  seven  attributes 
required  adjustment  according  to  this  particular  strategy.  The  weights  for 
submarine  information  parsimony  and  for  submarine  status  discriminabi 1 i ty 
are  not  displayed  since  they  remained  at  the  initial  value  of  unity 
throughout;  that  is,  the  information  preferences  in  this  strategy  did  not 
require  their  alteration.  Among  the  five  attribute  weights  that  did  adjust, 
the  pattern  is  pretty  much  the  same  across  trials.  After  considerable 
adjustment  during  tne  first  few  trials,  the  weights  reach  a plateau  where 
they  remain  stable  from  trial  3 through  7.  During  this  period  of  no 
adjustments  there  is  perfect  correspondence  between  the  model  and  the 
operator  in  predicted  and  chosen  information  sources,  respectively. 

After  the  seventh  trial  a new  environmental  situation  was  encountered 
in  the  scenario,  involving  the  potential  collision  of  the  submarine  and 
whale.  At  that  point  it  became  necessary  to  discriminate  between  the 
detection  of  the  two  objects  by  the  selection  of  a D sensor.  Since  this 
choice  was  not  correctly  predicted  by  the  multi-attribute  information  utility 
model  as  it  stood,  the  attribute  weights  were  consequently  adjusted  by  the 
model.  From  the  10th  trial  through  the  24th  no  environmental  situations 
occurred  which  had  not  been  faced  previously  in  the  tracking  session,  and  the 
model  was  again  able  to  predict  operator  information-selection  choices  with 
perfect  accuracy. 

The  general  direction  of  movement  of  the  attribute  weights  and  their 
terminal,  convergent  values  reflect  their  impact  on  the  overall  MAU  of  the 
selected  information  sources.  In  Strategy  II,  the  operator's  preferences 
for  high  expected  information  content  maintain  the  associated  attribute 
weights  at  high  levels.  Since  higher  attribute  levels  for  cost  indicate 
less  expensive  information,  his  preferences  for  higher  information  costs 
decrease  the  weights  of  the  cost  attributes  (except  when  faced  with  a 
collision,  where  he  prefers  lower  cost/reliability  information  which  is  more 
discriminatory  --  i.e.,  a D sensor  over  an  Ml  sensor).  When  a need  and 


preference  to  discriminate  between  a submarine  and  a whale  is  invoked  at 
trial  8 (i.e.,  collision  situation)  the  corresponding  attribute  weight 
rises  above  its  initial  value  of  one;  this  heightened  weight  contributes  to 
subsequent  correct  predictions  of  the  operator's  selection  of  information 
whenever  a collision  situation  is  re-enccuntered. 

4.3.3  Model  Performance  with  Changes  in  Strategy.  In  the  previous  section, 
it  was  demonstrated  that  the  multi-attribute  information  utility  model  can 
rapidly  learn  a fixed,  consistent  operator  strategy  after  starting  from 
scratch.  However,  one  of  the  major  functions  of  an  on-line  adaptive  model 
is  to  keep  track  of  an  operator's  behavior.  Such  a model  must  be  sensitive 
to  behavioral  changes  and  be  able  to  adjust  its  parameters  in  order  to 
remain  in  phase  with  the  operator  after  a limited  degree  of  lag.  In  the 
present  case,  behavioral  changes  take  the  form  of  shifts  in  information 
selection  strategy  that  might  occur  in  dynamic  tracking  environments. 

The  dynamic  response  of  the  model  was  tested  by  beginning  with  equal 
attribute  weights  and  having  the  operator  perform  the  tracking  task  according 
to  a predefined,  fixed  strategy  until  steady  state  behavior  of  the  model  was 
observed.  Once  the  model  was  accurately  predicting  (over  several  trials) 
the  strategy  actually  being  employed,  that  strategy  was  modified  in  some  way. 
When  the  modified  strategy  was  stabily  predicted,  another  change  in  informati 
preferences  was  implemented.  Thus,  a total  of  three  different  information 
selection  strategies  were  used  in  succession.  These  strategies,  which  are 
distinguished  by  the  type  of  information  preferred  for  tracking  the  submarine 
and  the  whale,  are  depicted  in  Table  4-2. 

The  attribute-weight  training  curves  are  displayed  in  Figures  4-4, 

4-5,  and  4-6.  The  curves  plot  the  changes  in  specific  attribute  weights  as 
a function  of  model  adjustments.  The  points  at  which  each  new  suhstrategy 
(either  for  tracking  the  submarine  or  the  whale)  is  adopted  are  indicated  by 
the  arrows  in  the  figures.  The  information  source  preferences  annotating 


TABLE  4-2.  SENSORS  PREFERRED  ACCORDING  TO  STRATEGIES 
UTILIZED  IN  DYNAMIC  ADAPTATION  EVALUATION 


Environmental 

Situation 

Intel  1 igence 
Report 

Characteristics 

Strategy 
1 2 3 

Submarine,  floating 

Pr  > 0,  PD  = 0 

Ml 

M2 

Ml 

or  resting 

r K 

Submarine,  floating 

PF  > 0,  PR  > 0 

H 

H 

Ml 

only 

r K 

- 

I 

Whale,  high  Pw  > .40  B D B 

probabi 1 i ty 


Whale,  low  Pw  > .40  B D D 

probabi 1 i ty 


Weight 
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FIGURE  4-4 

SENSITIVITY  OF  COST  WEIGHTS 
TO  CHANGES  IN  STRATEGY 
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Figure  4-4  shows  the  sensitivity  of  the  cost  weights  to  changes  in 
strategy.  To  illustrate  how  the  graph  might  be  read,  the  adaptation  of  the 
weight  for  the  submarine  information  cost  attribute  is  considered.  In 
Strategy  1 (H>B  when  Pp>0  and  PR=0,  *41 >M2  when  Pp  0 and  PR  0),  higher 
information  cost  is  consistently  preferred  and  thus  the  attribute  weight 
declines  from  its  initial  value  of  1 to  close  to  0.  Then  with  Strategy  2, 
higher  information  cost  is  preferred  in  one  m..:  marine  search  situation 
(H>B  for  Pp-0  and  PR=0),  but  lower  cost  is  preferred  in  the  other  submarine 
search  information  (M2>M1  for  Pp-0  and  P 0).  Thus  the  attribute  weight 
moves  up  and  down  until  eventually  stabilizing  after  about  55  adjustments. 
Finally,  in  the  third  strategy  where  the  most  expensive  sensor  (Ml)  is 
preferred  in  all  submarine  search  situations,  the  weight  level  shows  a 
steady  decline.  With  respect  to  the  other  curves  in  Figures  4-4,  4-5,  and 
4-6,  similar  analyses  can  be  made  concerning  the  changes  in  information- 
selection  irategy  and  their  impact  on  the  attribute  weights.  Overall, 
examination  of  the  data  reveals  that  it  usually  takes  the  model  less  than 
20  adjustments  to  catch  up  with  the  operator  in  terms  of  correctly  predicting 
his  new  information  preferences. 

4.3.4  Size  of  Adjustment  Step . It  is  obvious  from  Figures  4-4,  4-5,  and 
4-6,  as  well  as  from  Figure  4-3,  that  the  size  of  the  attribute  weight 
adjustment  step  varies  considerably  across  the  different  attributes. 

According  to  the  weight-training  rule  (Table  3-4),  the  size  of  an  adjustment 
depends  upon  \ (which  is  held  equal  to  0.4  across  all  attributes)  and  the 
difference  in  attribute  levels  between  the  operator-chosen  information  source 
and  the  model -predicted  source.  Thus,  the  variance  within  the  levels  that 
each  attribute  can  assume  impacts  upon  the  size  of  weight  adjustments. 
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Referring  to  Table  3-3,  it  is  seen  that  the  situation  mask  for  the  cost 
attributes,  is  either  0 or  1 , but  for  the  expected  information  content 
attributes,  the  mask  is  a probability  or  continuous  number  between  0 and  1. 
The  information  source  characteristic  levels  are  continuous  (between  0 and 
1)  for  each  of  these  two  types  of  attributes,  but  the  difference  in  the 
scales  for  the  situation  mask  account  for  the  relatively  steep  adjustments 
for  the  cost  attributes  and  relatively  shallow  adjustments  for  the  expected 
information  content  attributes  (Figures  4-3,  4-4,  and  4-5).  In  contrast, 
the  adjustments  for  the  submarine  information  parsimony  attributes  are 
steepest  of  all.  This  is  because  both  the  situation  mask  and  source 
characteristic  for  this  attribute  assume  a descrete  distribution  --  either 
0 or  1 , consequently;  the  attribute  level  can  take  a value  of  only  0 and  1 
resulting  in  the  highest  intra-attribute  level  variance. 


4 . 4 Automatic  Information  Selection 

Section  3.5  described  two  different  levels  of  information  selection. 
The  first  involves  the  selection  of  a single  information  source  in  each 
location  of  search  having  non-zero  object  probability.  The  second  process 
relates  to  the  further  selection  of  an  "optimum"  set  across  a number  of 
locations.  As  reported  below,  both  selection  processes  were  demonstrated 
in  the  empirical  evaluation  of  the  model. 


4.4.1  Information  Source  Selection.  The  mechanism  used  to  select  among 
competing  sources  within  each  location  of  ocean  was  a straight-forward 
application  of  the  MALI  principle;  namely,  the  source  giving  the  highest 
computed  MAU  value  was  selected.  Table  4-3  gives  two  examples  of  the 
computation  taken  from  an  actual  performance  run.  The  examples  represent 
two  different  intelligence  reports  (i.e.,  object  probabilities)  regarding 
the  subject  location.  The  calculation  is  made  during  the  latte*  part  of  the 
implementation  of  Strategy  II,  after  the  attribute  weights  had  converged  to 
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stable  values.  Both  examples  show  the  entries  for  the  situation  inask  vector 
corresponding  to  the  given  intelligence  report,  the  information  source 
characteristic  vector  and  computed  attribute  level  vector  for  two  sources 
with  the  highest  MAU  values,  the  attribute  weight  vector,  and  the  aggregate 
MAU  or  utility  for  the  two  competing  sources  considered.  In  Example  \,  the 
Ml  sensor  was  selected  as  the  information  source  with  the  maximum  MAU. 
Example  2,  the  D sensor  was  similarly  chosen. 


One  of  the  more  important  implications  of  employing  the  multi-attribute 
utility  model  to  automatically  select  among  competing  information  sources  is 
that  different  strategies  (i.e.,  attribute-weight  vectors)  will  lead  to 
different  distributions  of  selected  information.  Figure  4-7  illustrates  the 
relative  frequency  of  selection  for  the  five  possible  information  sources  after 
training  on  Strategy  I and  Strategy  II.  Each  histogram  is  based  on  a total 
of  about  150  separate  information  messages.  Under  Strategy  I,  more  than 
90%  of  the  automatically  selected  messages  consisted  of  H and  B type  sensors, 
each  represented  a little  more  than  45%  of  the  time,  with  the  remaining 
messages  contributing  a total  of  less  than  7 . Under  Strategy  II,  the 
frequency  distribution  is  roughly  the  complement.  The  Ml  and  D sensor  types 
are  each  selected  about  50%  of  the  time,  while  the  H and  B sensors,  which 
are  most  frequently  used  in  Strategy  I,  are  almost  never  selected. 

4.4.2  Information  Set  Selection.  The  second  level  of  information  selection 

was  based  upon  the  output  from  the  first  level  of  selection.  The  first-level  | 

output  is  a group  of  several  sources  (sensors),  each  providing  the  information 

] 

message  with  the  maximum  MAU  from  a separate  sector  of  search.  Taken  together 
for  a single  tracking  trial,  this  group  of  sensors  represents  a full  set  of 
information  messages  covering  all  locations  where  the  objects  of  search  may 
actually  be.  Since  some  of  these  messages  have  much  less  information  utility 
than  others,  the  MAU  principle  can  be  used  to  prune  away  messages  which 
contribute  relatively  little  to  the  overall  utility  of  the  information  set. 
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In  this  way,  the  operator  can  be  provided  with  a reduced  set  of  the  most 
valuable  information. 

The  pruning  rule  implemented  was  as  follows.  First,  all  the  members 
of  the  full  information  set  for  a trial  were  ordered  according  to  decreasing 
MAU.  It  had  been  previously  determined  that  because  of  the  tracking  task 
requirements,  the  three  messages  with  the  highest  MAU  must  always  be 
transmitted  to  the  operator.  The  selection  algorithm  then  added  messages 
to  this  minimum  set  according  to  the  following  instruction:  add  each  next 

highest  ranked  message  only  if  its  MAU  exceeds  15%  of  the  sum  total  MAU  of 
the  messages  already  in  thu  set.  Once  the  final  set  was  determined  all 
selected  messages  were  transmitted  simultaneously  to  the  operator.  The 
cut-off  point  of  15%  was  arrived  at  by  analysis  of  previous  data  (Freedy  et 
al.,  1976)  and  through  pilot  experimentation. 

Figure  4-8  demonstrates  the  effect  of  this  pruning  rule  in  reducing 
information  sets  generated  by  two  different  operator  strategies.  Strategies 
I and  II  were  each  run  for  22  successive  trials  taken  from  an  identical 
portion  of  the  ASW  tracking  scenario  (i.e.,  identical  trial -by-trial  movements 
of  the  submarine  and  whale).  The  convergent  set  of  attribute  weights  were 
frozen  for  each  strategy,  and  information  messages  were  automatically  selected 
and  pruned  according  to  the  15%  rule.  The  results  are  plotted  in  terms  of 
the  median  percentage  of  accumulated  utility  contributed  by  each  additional 
ranked  message. 

For  Strategy  I,  the  15%  cut-off  line  hits  the  accumulated  utility 
curve  between  the  third  and  fourth  ordered  message.  For  Stratety  II,  the 
15  line  hits  the  curve  between  the  fifth  and  sixth  messages.  Thus  more 
information  is  pruned  under  Strategy  II  than  under  Strategy  I.  Taking  into 
account  the  characteristic  preferences  of  each  strategy,  the  results  are 
entirely  reasonable.  Strategy  II  emphasizes  the  use  of  high  cost  sensors 
with  high  expected  information  content.  Therefore,  under  this  strategy. 
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most  of  the  MAU  in  the  full  information  set  is  concentrated  in  a few  locations 
with  the  balance  distributed  in  small  amounts  among  the  other  locations.  In 
contrast.  Strategy  I relies  on  sensors  which  are  less  costly  and  carry  less 
expected  information  content;  thus  it  generates  a more  homogeneous 
distribution  of  MAU  across  the  different  search  locations.  Under  Strategy 
I,  therefore,  a larger  set  of  information  messages  is  required  to  provide 
approximately  the  same  relative  level  of  incremental  utility  as  obtained 
under  Strategy  II. 

4 . 5 Operator  Performance  with  Automatic  Information  Selection 

4.5.1  Purpose.  The  demonstrations  described  thus  far  focused  on  model 
performance.  In  sum,  they  illustrate  that  the  model  can  select  information 
on  the  basis  of  adaptively-estimated  needs  and  preferences.  It  remains  to 
be  shown  that  an  operator  presented  with  information  automatically  selected 
in  accordance  with  a stable  preference  model  can  successfully  perform  the 
ASW  tracking  task.  This  demonstration  was  accomplished  using  a single 
expert  operator,  who  performed  the  ASW  task  under  several  conditions  of 
automatic  information  selection. 

4.5.2  Test  Conditions.  Information  was  automatically  presented  in  accord 
with  Strategies  I and  II.  For  each  strategy,  the  convergent  attribute  weights, 
previously  obtained  from  sessions  run  in  the  manual  mode,  were  input  into 

the  multi-attribute  information  utility  model.  The  model  was  then  employed 
to  select  information  and  present  it  automatically  to  the  operator  over  a 
set  of  24  task  trials;  during  these  trials,  movements  of  the  objects  were 
different  than  in  the  original  model-training  sessions.  Two  automatic 
selection  modes  were  tested:  (1)  presentation  of  the  full  information  set. 

and  (2)  presentation  of  a reduced  set  pruned  according  to  the  15  rule.  On 
each  tracking  trial,  the  operator  simultaneously  received  the  intelligence 
report  and  the  information  output  of  the  automatic  selector  mechanism.  On 
the  basis  of  these  data  alone,  he  made  an  object  status  report,  which 
generated  a new  intelligence  report  and  new  source  selection,  and  so  on. 
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In  addition  to  the  2x2  combination  of  demonstration  conditions,  i.e.. 
Strategy  (I  or  II)  x Information  Set  (Full  or  Pruned),  a control  treatment 
was  introduced  for  performance  comparison.  The  control  strategy  took  into 
account  the  information  needs  of  the  operator  (situation  mask  vector)  and 
the  differential  properties  of  the  sensors  (source  characteristic  vector), 
but  it  did  not  include  the  operator-initiated,  differential  preferences  for 
information  attributes.  That  is,  the  MAU  calculation  utilized  the  same 
attribute  level  vector  as  did  the  other  strategies,  but  the  attribute  weight 
vector  was  replaced  by  a unit  vector.  Thus,  each  attribute  was  accorded 
equal  importance  throughout  the  test  session,  and  attribute  weights  remained 
equivalent  to  the  initial  state  of  the  adaptive  model.  Data  collected  under 
autopiatic  information  selection  governed  by  equal  attribute  weights  was 
expected  to  reflect  baseline  performance  with  a strategy  characterized  by 
minimal  cost,  minimal  expected  information  content.  Because  fhe  information 
generated  by  the  control  strategy  was  of  such  low  overall  utility,  it  was 
not  meaningful  to  apply  the  pruning  rule  to  reduce  further  the  information 
set. 

4.5.3  Performance  Results.  The  results  of  the  performance  evaluation  are 
summarized  in  Table  4-4.  For  each  strategy  tested,  the  percentages  of 
submarine  and  whale  hits  are  listed  together  with  the  mean  cost  expenditure 
per  trial.  For  Strategies  I and  II,  the  performance  data  is  contrasted  for 
the  sessions  under  which  a full  or  pruned  information  set  was  presented  to 
the  operator.  In  comparing  any  two  conditions,  performance  in  terms  of  the 
hit  rate  is  generally  better  at  the  expense  of  higher  costs J For  example, 
more  than  four  times  the  costs  were  expended  to  achieve  a 92%  hit  rate  under 
Strategy  II  versus  a 58*  hit  rate  under  the  control  strategy.  Because  of 
this  trade-off  between  hits  and  costs,  it  was  useful  to  assess  overall  or 
net  performance  in  terms  of  an  effectiveness  index.  The  measure  employed 
was : 

'a  "hit"  is  a correct  detection  of  a submarine  or  whale. 
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TABLE  4-4 


OPERATOR  TRACKING  WITH 
AUTOMATIC  INFORMATION  SELECTION 


Utility-Based  Strategies 


Sel ection 
Criterion 


Full  Info 
Set 


Pruned 
Info  Set 


Strategy  I 

Strategy 

II 

Sub 

Whale 

Mean 

Sub 

Whal  e 

Mean  ' 

Hits 

Hits 

Info. 

Hits 

Hits 

Info 

% 

% 

Cost 

% 

% 

Cost 

75 

42 

25 

92 

92 

44 

Control 

Strategy 

Sub  Whale  Mean 

Hits  Hits  Info 

% % Ci 

58  58 


7 5 29  12  100  88 
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nrcrPTiiirurcr  runrv  - TRACKING  PERFORMANCE 
EFFECTIVENESS  INDEX  — iNF6WTI0N~C0~Sr~  ’ wh'  ' 


TRACKING  PERFORMANCE  = 2 [ACCURACY]^  + [ACCURACY]WHALE , and 


ACCURACY  = HITS  - * MISSES 

The  index  is  a benefit-to-cost  ratio  which  has  face  validity  because:  (1) 

the  operator  was  previously  made  aware  that  submarine  tracking  had  twice  the 
importance  of  whale  tracking;  and  (2)  accuracy  in  real-world  ASW  tracking 
tasks  is  often  interpreted  in  terms  of  hit  inus  misses. 


Since  the  effecti vei  ess  index  avera  lire  than  1.7  times  higher  for 
tracking  performance  under  Strategy  II  con'll--  with  Strategy  I,  only  Strategy 
II  was  selected  for  further  analysis  (Table  4-5).  Under  Strategy  II,  operator 
performance  with  the  pruned  information  set  resulted  in  a 1.5  times  better 
effectiveness  index  than  obtained  with  a full  information  set.  The  Strategy 
II  pruned  condition  represents  performance  under  full  model  capability  i,  : . e . , 
where  information  is  both  selected  and  pruned  in  accordance  with  articulated 
operator  preferences).  It  is  thus  desirable  to  compare  performance  in  this 
condition  with  performance  in  the  control  condition  (i.e.,  without  specific 
information-attribute  preferences  and  without  pruning'  These  performance 
ratios  are  shown  in  the  bottom  part  of  Tabl  • Although  Strategy  II  with 
pruning  expended  more  than  3 times  the  info-n-  t i on  costs  of  the  control 
strategy,  tracking  performance  was  nearly  6 times  better,  resulting,  overall, 
in  an  effectiveness  improvement  ratio  of  l.f  to  1 

4.5.4  Information  Acquisition  Time.  Informal  observation  -indicated  that 
automatic  information  selection  markedly  reduced  the  time  normally  required 
for  informaf ion  acquisition  in  the  simulated  ASW  task.  The  time  required  by 
th<  operator  to  manually  select  and  distribute  info  - on  sources  was  about 
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TABLE  4-5 


PERFORMANCE  EFFECTIVENESS  SUMMARY 


EFFECTIVENESS  MEASURES 


CONTROL 

STRATEGY 

II 

STRATEGY 

FULL  SET 

PRUNED 

TRACKING  PERFORMANCE 

48 

252 

276 

INFORMATION  COST 

10 

44 

32 

EFFECTIVE  INDEX 

4.8 

5.7 

8.6 

STRATEGY  II  PRUNED  VS.  CONTROL 

STRATEGY 

PERFORMANCE  RATIO 

5.8:1 

COST  RATIO 

3.2:1 

EFFECTIVENESS  RATIO 

1.8:1 
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50  seconds.  In  constrast,  the  model  automatically  selected  and  distributed 
the  sources  in  about  1 second.  The  throughput  ratio  of  automatic  to  manual 
[ selection  was  thus  on  the  order  of  50  to  1. 

4.5.5  Information  Filtering.  Overall,  the  adaptive  system  was  able  to 
select  automatically  about  5 to  7 highly  useful  messages  from  a potentially 
available  universe  typically  in  the  order  of  78,000  messages,  a filtering 
ratio  of  over  15,000  to  1.  This  value  was  calculated  as  follows.  Each 
trial  yielded  an  average  of  7 locations  for  which  there  was  a non-zero 
|l  probability  of  finding  a submarine  or  whale,  and  information  from  each 

location  could  be  obtained  from  one  of  five  sources.  Thus  the  potential 
message  universe  was  5'  = 78,125. 
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5.  DISCUSSION  AND  IMPLICATIONS 

5 . 1 Generality  of  the  Mul ti -Attribute  Utility  Model 

5.1.1  Demonstration  Approach.  More  often  than  not,  a specialized  task 
scenario  is  designed  whenever  a newly  developed  decision  aiding  system  must 
be  implemented  for  experimental  testing.  In  fact  it  has  been  said  that  a 
skeptic  might  argue  that  demonstrations  of  certain  decision-aiding  systems 
"merely  show  that  one  can  design  a simulated  task  in  which  it  helps  to  have 
machine  assistance"  (Slovic,  Fischhoff,  and  Lichtenstein,  1977).  In  the 
present  work,  however,  a different  course  was  taken;  that  is,  the  new  multi- 
attribute information  utility  model  was  applied  to  an  existing  ASW  simulation 
task.  The  ASW  scenario  had,  in  fact,  been  originally  designed  as  a test  bed 
for  the  outcome-based  ADDAM  model  (described  in  Section  2.5.3)  which  is 
conceptually  very  different  from  the  model  developed  here.  The  successful 
demonstration  of  the  implementation  of  the  new  model  to  a scenario  developed 
for  another  purpose  endorses  the  generality  of  the  model. 

5.1.2  Replacement  of  Human  Function.  The  model  developed  here  provides 
another  demonstration  of  how  a simple  linear  model  can  be  employed  to  replace 
an  important  human  function  within  a decision  system.  Such  models  have 
proved  successful  in  a variety  of  tasks  (e.g.  , judgmental,  decision  making, 
attitudinal,  perceptual)  and  applications  (e.g.,  military,  medical,  social). 
In  the  present  work,  the  application  was  extended  to  an  information  selection 
task  within  the  context  of  ASW  tracking.  Furthermore,  unlike  the  more 
commonly  used  methods  of  direct  elicitation  or  regression  analysis,  the 
linear  weights  employed  by  the  multi-attribute  information  utility  model  wen 
adaptively  estimated  from  on-line  behavior.  Certainly,  the  highly 

general izable  features  of  mu  I ti -attribute  utility  assessments  (e.g..  Brown, 
Kahr,  and  Peterson,  1974;  Edwards,  1977)  combined  with  an  adaptive  dynamic 
utility  estimation  technique  which  is  growing  in  application  depth  (e.g., 
Felsen,  1976)  provide  a tool  of  far  reaching  potential. 
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Because  of  the  generality  and  power  of  operator-based  models  or 
decision  rules,  it  is  important  to  ask  why  they  can  perform  as  well  or  better 
than  the  actual  operator  himself.  Several  investigators  have  suggested  that 
the  superior  performance  of  the  models,  linear  or  otherwise,  is  due  to  their 
ability  to  eliminate  or  reduce  "noise"  effects  in  the  subjective  weighting 
of  information  and  in  erratic  operator  responses.  For  example,  Bowman  (1963) 
described  the  filtering  process  as  follows: 


"Man  seems  to  respond  to  selective  cut.  1 -i  his  nvir  men ' ... 
particular  things  i ei  t latch  his  zt 

random  and  particularistic  corn} , net  rs  -at.  '■  e e tin  in.:  ted)  through 
the  use  of  decision  rules  incorporating  \ • fficients  derive  / from 
the  operator's  own  i*<  • irr-  ■ ■ ' ' " 

Dawes  (1971)  put  it  somewhat  differently  by  stating  that: 

"A  mathematical  model , by  its  > ry  ■ ■■  . Is  • bstra  ■ i > 
proa  ss  It  n Is;  l , if  - ' • • 's.  be) 

involves  following  valid  principles  bui  f Hi  then, 
these  valid  principles  will  b>  abstmeu  i by  f; . me  lei  --  as 
long  as  the  deviations  from  these  , nlroiplss  art  > st  systemati  -a! i 
related  to  the  variables  the  decision  "\zker  is  \ >. sidering.  " 


After  a critical  review  and  analysis  of  linear  models  in  decision  making, 
Dawes  and  Corrigan  (1974)  conclude  that  the  iccess  of  these  models  is  tied 
up  with  their  inherent  robustness  and  their  propriateness  for  many  specific 
applications.  It  would  seem  from  the  results  of  the  present  investigation 
that  the  process  of  information  selection  in  a C3  systei  nas  characteristics 
which  lend  themselvt'  to  this  type  of  modeling  approach  The  trick,  of 
course,  in  a C3  operation  or  any  other,  is  to  identif.  'he  attributes  of 
information  which  can  adequately  model  the  operate:  »•<•"«*■  ‘ and 

preferences  in  response  to  the  situation. 
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5 . 2 Advantages  of  the  Multi- A ttribut p Utility  Mode ]_ 

The  mul ti -attribute  information  utility  model  developed  in  this 
research  is  characterized  by  several  attractive  features.  These  features, 
which  are  itemized  below,  can  be  seen  as  advantages  which  endorse  the 
application  potential  of  the  model.  The  advantages  arise  out  of  the 
theoretical  structure  of  the  model,  especially  the  decomposition  property. 
However,  they  have  all  been  empirically  illustrated  to  some  degree  in  the 
demonstration  and  evaluation. 


5.2.1  Generali ty.  The  adaptive,  multi-attribute  model  for  information 
selection  holds  a considerable  amount  of  general i ty.  It  can  be  applied  in 
situations  where  information  messages  can  be  decomposed  into  a small  set  of 
manageable,  quantifiable  attributes  which  have  two  critical  characteristics. 
First,  they  must  be  locigally  related  to  the  situation-specific  information 
requirements,  that  is,  their  relevance  to  specific  situations  must  be  known. 
Second,  they  must  directly  impact  upon  a decision  maker's  choices  among 
competing  information  sources  or  messages.  Several  military  decision  making 
environments  have  already  been  demonstrated  to  fit  this  paradigm  (e.g., 
Coates  and  McCourt,  1976;  Hayes,  1964;  McKendry,  Er.derwick,  and  Harrison, 
1971;  Samet,  1975). 


5.2.2  Parsimony.  The  model  is  parsimonious;  it  need  only  assess  an  operator's 
weights  for  a limited  number  of  information  dimensions  or  attributes.  Besides 
significantly  minimizing  the  model's  computational  needs  and  software 
complexity,  this  feature  is  in  consort  with  the  results  of  psychological 
experiments  (e.g.,  Hayes,  1964;  Slovic,  1975:  Wright,  1974)  and  contemporary 
decision  theory  (e.g.,  Tversky  and  Kahneman,  1974);  namely,  that  a decision 
maker  can  perform  an  intuitive  conscious  weighting  and  aggregation  of  only 
a relatively  small  number  of  what  he  considers  to  be  the  important  dimer  ions 
common  to  the  decision  alternatives.  Furthermoi.  , when  decisions  are  based 
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on  a manageable  number  of  information  dimensions,  they  are  easier  to 
communicate  and  rationalize  --  especially  in  group  decision  making  situations 
(Gardiner  and  Edwards,  1975). 

5.2.3  Robustness.  Lii<e  other  linear  composition  mooels,  the  mul ti -attribute 
information  utility  model  is  robust;  that  is,  its  performance  (i.e., 
capability  to  mimic  the  information  selection  behavior  of  an  operator)  is 
not  signi f icant Jy  degraded  by  proportion -te1 y small  perturbations  in  the 
model's  parameters  (Dawes  and  Corrigan,  1974).  Such  robustness  probably 
contributes  to  the  finding  that  mul ti -attribute  utility  assessment  techniques 
have  proved,  in  certain  instances,  to  be  more  reliable  and  valid  than  direct, 
holistic  assessment  procedures  (Newman,  1975;  Samet,  1976). 

5.2.4  Speed  of  Adaptation.  The  adaptive  model  adjusts  all  parameters  with 
each  incorrectly  predicted  operator  decision  (i.e.,  information  selection). 
Thus,  weights  (utilities)  for  specific  information  attributes  can  be  trained 
rapidly  during  sessions  in  which  the  operator  chooses  information  manually. 

5.2.5  Flexibility.  The  mul ti -attribute  ut i i ty  model  is  inherently  flexible. 
If  accuracy  of  prediction  of  information  selection  behavior  is  not  sufficient 
(i.e.,  if  attribute  weights  cannot  be  trained  to  stable  values),  additional 
features  or  attributes  can  be  added  and  inappropriate  ones  deleted.  The 
response  to  dynamic  changes  in  conditions  is  . i i 1 a rl y flexible.  In  instances 
where  conditions  change  rapidly  and  radically,  new  sets  of  weights  trained 
for  the  conditions  can  be  substituted.  Such  weight  vectors  could  be 
previously  trained  either  in  actual  operational  situations  or  in  step-through 
simulations. 

5.2.6  Versati 1 ity.  The  model  can  he  applied  in  u variety  of  situations, 
involving  deterministic  as  well  as  probabilistic:  environments.  In  the 
probabi 1 istic  situation,  an  attribute  miqht  not  be  present  at  all  in  some 
information  messages,  or  it  might  "ccur  with  le  known  probability. 


Whatever  the  case,  the  modeling  equation  allows  for  any  attribute  level  to 
either  be  zeroed  out  (i.e.,  be  made  irrelevant)  or  multiplied  by  the 
probability  that  the  attribute  will  be  present,  giving  a measure  of  expected 
attribute  presence. 

5 . 3 Supervision  of  Information  Flow  in  C3  Systems 

As  stated  in  the  Introduction,  development  of  the  present  adaptive 
multi-attribute  utility  model  represents  one  step  toward  development  of  a 
supervisory  set  of  adaptive  programs  to  control  overall  information  flow  in 
C3  systems.  This  section  outlines  a method  for  C3  system  description,  and 
suggests  in  brief  how  the  present  model  might  contribute  to  the  supervisory 
control  function.  Further  development  of  the  supervisory  concept  will  be 
the  subject  of  subsequent  project  reports. 

5.3.1  C3  System  Description.  We  can  consider  the  typical  C3  system  as  a 
hierarchical,  multi-level  arrangement  of  users.  Figure  5-1  illustrates  the 
general  configuration.  People  at  one  level  process  information  for  people 
at  the  next  level,  collecting  and  integrating  data  until  a decision 
commensurate  with  their  level  can  be  made.  Thus  each  person  in  the  structure 
is  at  times  a user  of  information,  at  times  a source  of  unprocessed  or 
processed  information,  and  at  times  a source  of  decisions  passed  to  higher 
levels  of  the  hierarchy.  The  idea  that  information  transmission  in  such  a 
system  could  be  expressed  in  matrix  form  was  advanced  by  Thornton  Roby  (1968), 
whose  untimely  death  prevented  his  further  development  of  this  concept.  We 
can  illustrate  the  methodology  by  considering  a prototypical  ASW  command 
team,  consisting  of  people  (A,  B,  and  C),  each  of  whom  tends  to  focus  on 
information  from  the  outside  world  relating  to  certain  specific  aspects  of 
the  ASW  picture,  and  each  of  whom  is  responsible  for  certain  types  of  ASW 
decisions. 
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FIGURE  5-1 

PARTIAL  REPRESENTATION 
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Figure  5-2  exemplifies  the  steps  involved  in  constructing  the  overall 
information  transmission  matrix  (HRN)  from  the  constituent  matrices  (H,  R 
and  N)  for  this  illustrative  situation.  In  matrix  H,  the  column  headings 
(i,  j,  k,  represent  classes  of  information  (e.g.,  intelligence,  weapons 

characteristics,  sensor  returns,  equipment  ava i labi 1 i ty , etc.),  while  the 
row  headings  (0,  P,  Q)  represent  information  sources.  A unit  entry  in  any 
row  indicates  that  the  source  is  a producer  of  information  in  the  associated 
class.  Column  headings  (w,  x,  y,  z)  in  matrix  R represent  classes  of 
decisions  (e.g.,  target  location,  sensor  allocation,  weapons  readiness,  etc.). 
Cell  entries  in  R give  the  relevance  of  a given  class  of  information  to  a 
certain  decision.  Finally,  matrix  N relates  personnel  to  class  of  decisions; 
in  this  simple  case,  person  A is  responsible  for  w decisions,  B for  y 
decisions,  and  C for  x and  z decisions.  Matrix  multiplication  of  H and  R 
yields  matrix  HR  as  an  interim  product,  this  matrix  indicates  the  contribution 
of  each  information  source  to  the  output  decisions.  Matrices  HR  and  N are 
multiplied  to  obtain  the  information  transmission  matrix  HRN.  This  matrix 
relates  sources  of  information  to  users  of  information,  cell  entries  are 
information  classes,  as  mitigated  by  the  decision  needs  of  the  system. 

In  this  simple  example,  only  outside  sources  were  considered,  and 
users  acted  only  as  users.  Since  users  can  also  act  as  sources  to  other 
users,  the  next  step  of  complexity  would  add  another  dimensions  to  the 
transmission  matrix,  i.e.,  it  would  become  a three-dimensional  matrix, 
relating  outside  sources,  users,  and  inside  sources.  Other  dimensions  are 
also  possible,  and  the  procedure  would  have  to  be  adjusted  to  handle  multipie 
sources  for  information  items.  Digital  computers  are  well  suited  to 
manipulation  of  mul ti -dimensional  matrices,  and  reasonable  expansion  of  size 
should  pose  no  immediate  problems.  The  most  important  refinement,  however, 
lies  in  replacing  the  basic  0 and  1 matrix  entries  with  numerical  weights 
which  reflect  the  value  of  that  connection  in  the  information  system.  The 
overall  value  of  a constituent  C3  link,  then,  would  be  determined  by  a 
combination  of  weights  through  the  matrix.  Optimum  information  flow 
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through  the  system,  i.e.,  maximum  true  value,  would  be  achieved  .when  the 
weights  are  adjusted  in  accord  with  the  immediate  requirements  of  the  users 
and  of  the  situation. 

The  supervisory  computer  program  constitutes  the  mechanism  by  which 
the  weights  of  the  elements  in  the  communications  matrix  are  adaptively 
adjusted.  It  is  suggested  that  the  program  contain  both  heuristic  control 
algorithms,  which  are  situation-dependent,  and  a set  of  behavioral  models, 
which  depend  on  psychological  constructs  and  on  individual  user  characteristics. 
Together,  these  determine  the  instantaneous  element  weights.  Among  the  most 
significant  models  are  those  which  define: 

(1)  Multi-Attribute  Utility 

(2)  Information  Routing 

(3)  Information  Pacing  and  Load 

Feasibility  of  the  first  model  has  been  demonstrated  by  the  present  work. 
Application  of  this  work  to  realization  of  the  other  two  models  is  discussed 
briefly  below. 

5.3.2  Information  Routing.  The  multi-attribute  utility  model  combines  well 
with  the  communication  matrix  techniques  suggested  by  RuOy  (1968)  and 
described  above.  In  essence,  the  matrices  < late  information  sources  to 
information  classes  or  attributes,  attributes  to  types  of  decisions,  and 
decision  types  to  users.  Ry  multiplying  the  matrices  together,  communication 
connections  are  enumerated  between  sources  and  users.  To  establish  the 
strength  of  connections,  the  information  attribute  weights  could  be  determined, 
and  be  adjusted  on-line  by  an  adaptive  model.  In  this  way,  the  model  could 
be  implemented  to  build  upon  the  application  of  the  communication  matrix 
concept  in  order  to  provide  a degree  of  automatic  routing.  In  addition, 
since  the  model  keeps  track  of  a separate  weight  for  each  information 


FIGURE  5-2 

MATRIX  REPRESENTATION  OF  A SIMPLE  C3  NODE 
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attribute,  it  ha‘  the  capability  to  deliver  rich  feedback  to  the  operator. 

For  example,  he  could  be  informed  of  the  relative  weights  he  is  placing  on 
such  attributes  as  information  cost,  reliability,  relevance,  and  if  desired, 
he  could  be  given  associated  corrective  instruc*  influence  his  manual 

routing  behavior. 

5.3.3  Pacing  and  Load.  According  to  the  information-selection  strategies 
examined  in  the  present  study,  the  op  w ling  with  a pruned  information 

set,  as  opposed  to  a larger  information  rformed  better  in  term:  of 

his  net  effectiveness  as  a de  akei  h this  result  was 

demonstrated  for  a single  operator  performing  in  only  one  session  for  each 
condition,  it  is  consistent  with  previous  findings  that  an  increase  in 
information  density/detail/load  is  not  r essarily  accompanied  by  an 
improvement  in  decision  quality  (Granda,  J Hayes,  1964;  Huber  et  al., 

1975;  Schroeder  and  Benbasat,  1975). 


The  consistency  of  experimental  ndings  concerning  the  effects  of 
information  load  on  information  proci  performance  have  led  investigat' 

to  postulate  an  inverted-U  relationship  between  the  variables  (Hogarth,  1975; 
Schroeder,  et  al.,  1967);  that  is,  performance  falls  off  both  when  information 
load  is  too  low,  and  when  information  load  is  too  high.  The  reasons  for  the 
fall-off  are  different,  however.  When  load  is  too  low,  the  operator  has 
insufficient  information  to  decide  optimal  . Wren  load  is  too  high,  he  has 
more  information  than  he  can  handle  mem 


It  would  require  systematic  experimentation  to  : fine  the  optimal 
information  load  which  would  maxi  lize  ASW  operator  performance  in  the  present 
task.  However,  performance  results  support  the  possibility  that  the  full 
information  set  exceeded  the  amount  of  information  ^hat  he  could  appropriately 
assimilate,  especially  in  light  ot  the  fact  that  the  sensor  outputs  were  of 
imperfect  reliability  and  often  cc> ■ f 1 acting,  in  the  i resent  case,  by  applying 


the  pr 


the  15%  pruning  rule,  the  size  of  the  information  set  was  apparently  reduced 
to  a more  intellectually  manageable  size. 

The  important  general  implication  that  emerges  is  that  if  the  optimal 
information  load  can  be  calibrated  for  a given  operator,  then  an  adaptive, 
on-line,  multi-attribute  information  utility  model  such  as  developed  here 
could  be  applied  to  select  the  appropriate  amount  of  most  useful  available 
information.  More  specifically,  one  input  to  the  multi-attribute  model 
would  be  the  immediate  circumstances  of  the  operator  or  recipient  of 
information.  That  is,  one  attribute  of  incoming  i iformation  would  be  defined 
as  the  current  information  load  or  message  rate.  If  the  load  is  high,  the 
utility  weight  for  load  would  act  to  devalue  new  information.  On  the  other 
hand,  if  his  load  is  low,  the  load  weight  would  act  to  increase  his  overall 
utility  for  new  information.  By  requiring  information  to  exceed  a certain 
utility  threshold  before  being  presented,  the  model  could  include  the 
capability  to  adaptively  pace  information.  It  is  planned  to  explore  this 
approach  in  our  future  work. 
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